You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Manychat is a leading Chat Marketing platform. We enable businesses and creators
22
-
to drive more sales and conversions on messaging apps, such as Instagram,
23
-
WhatsApp, Facebook Messenger, and Telegram, using automation.
21
+
**We’re [Manychat](https://manychat.com/)! 👋🏻 Chat automation for 1M+ businesses at massive scale, with Python infrastructure that actually handles it without headaches.**
24
22
25
-
Trusted by over 1 million brands in 170+ countries, we're an official Meta
26
-
Business Partner, backed by top investors, including Bessemer Venture Partners.
23
+
Bet you know this feeling all too well: deploying an LLM feature and HOPING for the best in production 🤞🏻
27
24
28
-
Our global team of 350+ people is located around international offices in
29
-
Austin, Barcelona, Yerevan, São Paulo, and Amsterdam. 🌏
25
+
The costs, rate limits, provider outages taking down your entire feature… it used to haunt us, but we developed the systems that can handle it all.
30
26
31
-
Founded in 2016, we still embrace a startup work culture — making fast
32
-
decisions, setting ambitions goals, fostering open feedback and transparency. We
33
-
think big and win together. 🏆
27
+
We power chat automation for over a million creators and brands, so "the LLM is down" isn't an acceptable answer, ever. We built infrastructure that expects LLMs to misbehave:
28
+
- Multi-provider routing that fails over from Azure to OpenAI mid-retry.
29
+
- Weighted traffic distribution that we can rebalance without deploying code.
30
+
- Cooldowns that pull failing backends out of rotation automatically.
31
+
- Observability that actually helps — Prometheus metrics, Grafana dashboards, OpenTelemetry traces with business context so we know which customer's request just broke.
34
32
35
-
We value our team, offering competitive compensation and benefits, and a hybrid
36
-
work model to split time between home and cozy offices.
33
+
We also monitor our asyncio event loop like hawks, because nothing ruins your day faster than discovering a blocking call is starving your entire service.
37
34
38
-
We care intensely about our customers and their goals. By experimenting a lot
39
-
and testing hypotheses, we continuously improve our product to help businesses
40
-
grow and generate revenue on their social channels. That’s how we deliver
41
-
results and dazzle our customers. ⚡️
35
+
**And we'd love to discuss this and all things Python with like-minded people!** Come by our booth to talk to the engineers who built this infrastructure. We can tell you how we handle rate-limit cascades. How we track token costs per AI agent. What it's like to debug distributed traces when you're trying to figure out why one specific request took 6 seconds instead of 2. And anything else you'd like to know about working in such a high-load environment.
36
+
37
+
Come see us on stage:
38
+
- Daria Korsakova, Python Engineer at Manychat will talk about Practical observability for Python APIs, workers & jobs
39
+
- Sergi Porta, Python Team Lead at Manychat will talk about LLM Traffic Spikes: Routing, Rate Limits, and Failover in Python
40
+
41
+
Read the full technical breakdown of our infrastructure: https://medium.com/manychat-engineering/how-to-survive-llm-traffic-spikes-in-python-73955ee9426f
0 commit comments