About Crisp

Crisp consolidates, processes and activates every byte of data for food suppliers, retailers, and distributors for better forecasting and ideal production.

About The Role

Here at Crisp, we value the strength in teamwork and strongly believe that it’s the key to Crisp’s success. By bringing together bright, motivated creators, wherever they live and work, we leverage humanity’s diversity of experience and background to address the challenges facing our food supply and solve them together. Come join us and help build the type of business you'd like to be a part of. Crisp is a socially conscious, distributed team. We offer you the opportunity to solve global food industry challenges while living where you’re most comfortable and contributing to the community you belong to. We believe in transparency, diversity, and merit, fostering a culture of empowerment, personal impact, and career growth.

What You’ll Do

As a Web Data Extraction Specialist at Crisp, you will play a critical role on our data ingestion team. Your primary responsibility will be to develop and maintain automated web scrapers and API integrations using Python and automation tools (e.g., Selenium, Playwright), ensuring seamless and reliable data extraction from various retailer and distributor portals. You will contribute to proprietary automation frameworks, monitoring systems, and CI/CD pipelines, optimizing our data extraction operations' scalability and resilience.

What You’ll Bring

Qualities of an ideal Web Data Extraction Specialist candidate:

• Expertise in Web Scraping: Demonstrated proficiency in building resilient web scrapers using Python and automation frameworks such as Selenium or Playwright. Experience with navigating multi-factor authentication, CAPTCHAs, and anti-bot measures.
• API Utilization: Experience discovering and integrating with (undocumented) APIs to streamline and enhance data extraction operations.
• Robust System Design: Adept at implementing comprehensive error handling strategies, retry logic, and efficiency improvements.
• Cloud Technologies: Experience with Google Cloud Platform services including Cloud Storage, Cloud Run, BigQuery, Cloud Composer (Airflow), Secret Manager, and logging/monitoring solutions.
• CI/CD Proficiency: Comfortable using Docker, GitHub Actions, and DevOps practices to streamline deployments and improve system reliability.
• Problem-Solving Aptitude: Strong analytical skills with a capability to independently troubleshoot complex issues in remote environments. Additional preferred skills:
• Knowledge of distributed web scraping, proxy management, and scalable crawling solutions.
• Understanding of database optimization and performance enhancement for data pipelines.
• Familiarity with monitoring and alerting systems like Google Cloud Monitoring and Datadog.
• Experience with data engineering processes, including dbt, SQL transformations, and ETL methodologies.
• Experience working in an organization that practices secure software development with knowledge of credential and secret management best practices. Signs of a great candidate for Crisp:
• Collaborative: Actively leverages colleagues' perspectives to drive success and contributes to collective growth.
• Customer Focused: Prioritizes the customer voice, ensuring solutions consistently meet their needs.
• Ambitious, Curious, and Resourceful: Demonstrates natural curiosity, driven to self-manage and deliver impactful results.
• Disciplined and Reliable: Thrives in a distributed work environment, consistently meeting commitments, and proactively communicating challenges.
• Appreciative of Honest Feedback: Values constructive feedback as opportunities for improvement and growth.
• Efficiency-Oriented: Identifies and automates routine tasks to maximize productivity and impact.