Business intelligence & analytics play important roles in the tech industry, especially in data-driven companies like Facebook. As the company evolves, different business teams have their own ways of building their business reports. Because data demands also grow rapidly, teams face similar strategic and technical challenges in terms of data consistency, efficiency of data delivery and data security.
The data delivery platform is targeted to address these challenges. We use query templates and the definition of metrics to solve the data consistency problem, and the Spark SQL-based caching service to solve the data delivery efficiency problem, with security naturally integrated into the metric definition.
演讲提纲
- Same metric has different interpretation in different business domains
- Data queries live in everywhere in codebase
- Surfacing up to TB of data to dashboards in real time
- Data security (access control to dashboards/reports)
- The data delivery platform
- Overview
- The framework
- Query template & Metric engine
- SQL Caching
- Data Security
参考翻译:
商业智能和商业分析(Business Intelligence & Analytics)在技术领域扮演着重要的角色,尤其是在像Facebook这样的数据驱动型企业。随着公司不断的发展壮大,不同的商业团队根据自身需求和方式进行商业报告,因此,伴随着大家对数据的需求越来越大,在数据一致性、数据查询效率和数据安全层面上,团队们面临着相似的战略规划和技术挑战。
这个数据递送平台 (Data Delivery Platform) 就是为了对症下药解决这些挑战而建立的,我们团队使用查询模版和Metrics定义来解决数据一致性问题,利用Spark SQL 解决数据分发效率的问题,与此同时,安全性是自然而然贯穿到metric定义中的。
演讲提纲
- 同样的Metric在不同的业务领域有不同的解释
- 代码库要承担高强度的实时数据查询
- 将多达TB级数据实时呈现到数据报表
- 数据安全(对数据报表的访问控制)
- 数据递送平台 (Data Delivery Platform)
- 概览
- 整体框架
- 查询模版 & Metric引擎
- SQL 缓存
- 数据安全
听众收益
- 在中型之大型公司数据规范化的重要性以及如何做到。
- 如何灵活并选择正确的技术解决相应的数据难题。
- 如何用正确的架构把每一个解决方案模块整合起来。