Shourya Gupta

• Built Go and Python systems that modeled real-world engineering workflows for use in advanced LLM training.

• Created multi-turn conversational datasets by simulating realistic developer–AI interactions.

• Reviewed and corrected LLM generated code, ensuring correctness, security, and best engineering practices.

• Identified weaknesses in LLM responses and produced corrective feedback loops to improve model behavior.

• Refactored large open-source codebases for performance, improving their suitability as training data.

• Led backend engineering and system design of the platform helping TikTok creators grow with personalized content recommendations and track their growth by providing detailed analytics.

• Developed event-driven FastAPI services using RabbitMQ and Redis, across analytics, recommendations, notifications, search, authentication (JWT and OAuth 2.0)

• Built a payments and subscription system, integrated NMI payment gateway, and merchant APIs.

• Implemented Redis caching to cut API latency by 60% and pub/sub to power real-time analytics via SSE.

• Optimized MongoDB schemas and improved query performance by almost 40% by adding compound indexes, reducing document size and also set up read replicas for high read throughput.

• Implemented Pydantic models to enforce data validation and consistency between request/response schemas.

• Built a Qdrant-based recommender for 50M+ videos, serving personalized content suggestions to creators.

• Designed data mining infrastructure processing 1M+ TikTok profiles daily using Celery, RabbitMQ, Redis, with retries, logging, and monitoring via Flower.

• Set up dockerization and CI/CD with Github Actions, reducing deployment time by 80%.

• Integrated monitoring and observability using ELK and BetterStack, reducing downtime by 70%.

• Created 1000+ custom web scrapers across e-commerce, fashion, travel, real estate, marketplaces, and social media platforms for 200+ diverse clients, achieving 98% client satisfaction rate

• Built web dashboards using React to visualize scraped data for clients, featuring trend analysis charts, and exportable reports, enabling stakeholders to monitor business metrics and make data-driven decisions

• Developed price monitoring system for Amazon using Playwright, capable of tracking 100K+ products daily

• Built robust ETL data pipelines using Pandas and custom validation Python scripts

• Built crawlers for social media websites such as Instagram and X (Twitter) for extracting targeted URLs while maintaining 100% platform compliance

• Designed distributed task queue systems using Celery framework and RabbitMQ with intelligent worker management for horizontal scalability and fault tolerance across data centers, handling 10M+ tasks daily

• Successfully reverse-engineered 150+ complex APIs and implemented advanced bot protection bypass techniques including CAPTCHA solving using Tesseract and OpenCV, browser fingerprinting, custom headers, cookies implementation, and session management

• Deployed comprehensive monitoring using Elasticsearch, Logstash, and Kibana (ELK Stack) for real-time system health and performance tracking