Data Engineer (Spark)
Latin America
Engineering /
Full-time /
Remote
Who we are:
Factored was conceived in Palo Alto, California by Andrew Ng and a team of highly experienced AI researchers, educators, and engineers to help address the significant shortage of qualified AI & Machine-Learning engineers globally. We know that exceptional technical aptitude, intelligence, communication skills, and passion are equally distributed around the world, and we are very committed to testing, vetting, and nurturing the most talented engineers for our program and on behalf of our clients.
We are seeking a highly skilled Data Engineer to join our team, focusing on developing, refining, and maintaining robust real-time and batch data infrastructures. The ideal candidate will have a strong background in SQL, Python, and Spark, with a passion for ensuring data quality and optimizing data workflows. #LI-Remote
What you will be doing:
- Develop, enhance, and manage both real-time and batch data infrastructures.
- Apply your expertise in SQL, Python, and Spark, alongside technologies such as HDFS, Snowflake, Hive, HBase, Scylla, Django, and FastAPI.
- Implement processes to guarantee the highest standards of data quality and accuracy across all production data systems.
- Collaborate with Data Engineers to optimize data models and workflows, ensuring efficiency and scalability.
- Develop robust ETL processes for comprehensive analysis and reporting to support data analytics processes.
- Partner with Product Managers to design and build innovative data products that drive business value and cooperate with the DevOps team to scale and optimize data infrastructure, ensuring reliability and performance.
- Participate in architecture discussions, influence the technology roadmap, and take ownership of new projects and initiatives.
What you must bring:
- Proven experience in developing and managing real-time and batch data infrastructures.
- Strong proficiency in SQL, Python, and Spark.
- Experience across the entire software development lifecycle, from inception to production and monitoring.
- Demonstrated ability to ensure data quality and accuracy in production environments.
- Familiarity with cloud services such as AWS, GCP or Azure.
- Excellent problem-solving skills and the ability to lead projects from conception to implementation.
Nice to have:
- Experience with large-scale distributed systems.
- Expertise in designing and architecting distributed low-latency and scalable solutions in either cloud or on-premises environments.
At Factored, we believe that passionate, smart people expect honesty and transparency, as well as the freedom to do the best work of their lives while learning and growing as much as possible. Great people enjoy working with other passionate, smart people, so we believe in hiring right, and are very selective about who joins our team. Once we hire you, we will invest in you and support your career and professional growth in many meaningful ways. We hire people who are supremely intelligent and talented, but we recognize that intelligence is not enough. Perhaps more importantly, we look for those who are also passionate about our mission and are honest, diligent, collaborative, kind to others, and fun to be around. Life is too short to work with people who don’t inspire you.
We are a transparent workplace, where EVERYBODY has a voice in building OUR company, and where learning and growth is available to everyone based on their merits, not just on stamps on their resume. As impressive as some of the stamps on our resumes are, we recognize that human talent and passion exist everywhere, and come from many backgrounds, so stamps matter much less than results. All of us are dedicated doers and are highly energetic, focusing vehemently on execution because we know that the best learning happens by doing. We recognize that we are creating OUR COMPANY TOGETHER, which is not only a high-performing fast-growing business, but is changing the way the world perceives the quality of technical talent in Latin America. We are fueled by the great positive impact we are making in the places where we do business, and are committed to accelerating careers and investing in hundreds (and hopefully thousands) of highly talented data science engineers and data analysts.
In short, our business is about people, so we hire the best people and invest as much as possible in making them fall in love with their work, their learning, and their mission. When not nerding out on data science, we love to make music together, play sports, play games, dance salsa, cook delicious food, brew the best coffee, throw the best parties, and generally have a great time with each other.