The Huawei India R&D Center has confirmed that the company has achieved a breakthrough in big data domain through its industry collaboration initiatives. It is believed that the India R&D center in Bengaluru has contributed to CarbonData project which was formally incubated to Apache Software Foundation by Huawei in June 2016.
According to Huawei, Apache CarbonData (incubating) is a fully indexed columnar and Hadoop native data-store for processing detailed queries and heavy analytical workloads on big data. Carbon Data has been designed and developed from the base platform to provide quicker response for different user queries.
It is columnar and supports Global Dictionary with multi-level indexing, deferred decoding and column groups making it unique as well as quick. It has a deep in hand integration with Spark, thus enabling spark users to readily use it. As most organizations use different query engines and data-stores and to meet their requirements of Interactive Analysis Queries (Multi-Dimensional OLAP), batch Queries (Big scan) and Operational Queries (Random Access/Narrow Scan) which results in replication of data and processing, thereby increasing the cost. So, Huawei looks forward at this one of the key challenges that CarbonData intends to solve.
Huawei India R&D center has identified the need for contribution and collaboration to open source years ahead of inception and has been playing an active role in executing Huawei’s open source strategy. The R&D center has contributed to open source projects in NFV, SDN, Cloud, Big data and analytics domains, and with the support of strong local eco-system is driving improvements in various domains through community cooperation and collaboration.
