While enforcing a consistent schema is reasonable for business events with well-defined structures, for logging this leads to significant reduction in developer productivity because log schema organically evolves over time. Incompatible field types cause type conflict error in ES, which drops the offending logs. ES (Elasticsearch) infers schemas automatically, keeps it consistent across the cluster, and enforces it on following logs. Log schema: Our logs are semi-structured.Both surfaced a lot of challenges to the ELK based platform, which were not visible at lower scale: In the past few years, the organic logging traffic growth resulted in a massive platform deployment size and the user requirements evolved significantly too. As we reached a scale bottleneck to support this rapidly growing traffic, we decided to take our insights about Uber’s many logging use cases and build our next-gen platform to fundamentally improve the reliability, scalability, performance, and most importantly to ensure a pleasant experience for both its users and operators. Since starting with ELK for logging in 2014, our system traffic volume and use case variance had both grown significantly. Right now, the platform is ingesting millions of logs per second from thousands of services across regions, storing several PBs worth, and serving hundreds of queries per second from both dashboards and programs. The logs are tagged with a rich set of contextual key value pairs, with which engineers can slice and dice their data to surface abnormal or interesting patterns that can guide product improvement. At Uber, we provide a centralized, reliable, and interactive logging platform that empowers engineers to work quickly and confidently at scale.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |