Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Tengyun driving and counting! Tianyi cloud TeleDB for AnalyticDB helps enterprises to build several positions efficiently!

2024-06-19 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > IT Information >

Share

Shulou(Shulou.com)12/24 Report--

With the advent of the intelligent era, the amount of global online data is growing rapidly. Enterprises hope to obtain valuable information from massive data, which makes the demand for data analysis growing. However, traditional databases and database products based on Hadoop architecture are difficult to meet the rapid growth needs of enterprises, and the industry is in urgent need of an analytical database (data warehouse) with excellent performance in terms of performance, economy, security and reliability.

To meet the market demand, Tianyi Cloud launched a new generation of cloud native analytical database TeleDB for AnalyticDB (referred to as TeleDB-ADB), which provides a new solution for the domestic data analysis market. At present, TeleDB-ADB has been applied in production scenarios such as data analysis and inventory query within China Telecom to help users organize, store and manage data efficiently.

TeleDB-ADB consists of three independent scalable components from top to bottom: metadata, computing cluster and storage layer. Based on the design of independent scalable components, TeleDB-ADB can handle the storage and management of very large-scale data and high-concurrency and high-performance data query requirements. The functions of each component are as follows:

Metadata: based on distributed KV design, it is responsible for processing and saving system metadata, such as table structure, index, partition information, data type, etc., and can manage hundreds of millions of database objects.

Computing cluster: MPP Shared-Nothing architecture is adopted, which allows multiple users to access the common data of the storage layer through different computing clusters, and perform high-performance loading, query and other work. Because the compute layer does not store user data and metadata, the component is stateless and can be provisioned on demand

Storage layer: cloud-based object storage system, which can access a large number of structured and semi-structured data, share data to global computing clusters, and write data in different clusters can be immediately visible in other clusters. It effectively avoids the additional impact and resource consumption caused by multi-cluster data redundant storage and data migration.

In addition to the above components, TeleDB-ADB is also equipped with efficient cluster management and control, the management and control cluster adopts graphical configuration, and the management database provides the capabilities including operation and maintenance management, rights management, monitoring and alarm, parameter configuration, and so on. Through the Raft consistency algorithm provided by ETCD, the consistency problem of key configurations in the whole database distributed system is solved. Relying on this design method, TeleDB-ADB can better support complex data analysis tasks, improve data processing efficiency and accuracy, as well as scalability, enabling users to create, delete and scale computing clusters on demand. At the same time, resources between clusters are completely isolated, and users can deploy hardware with different configurations, brands and architectures on private cloud platforms or physical servers.

TeleDB-ADB focuses on the four core elements of "performance, cost, flexibility and security", providing users with services such as massive data analysis and dealing with complex computing tasks.

Excellent performance, industry-leading

TeleDB-ADB adopts advanced distributed architecture, can easily deal with PB-level massive data, and has a data import rate close to GB/s. Whether structured data, semi-structured data or unstructured data, TeleDB-ADB can achieve efficient storage and query.

Among the 99 query SQL sets under the standard test dataset of TPC-DS (Analytical Database performance Test benchmark), the average performance of TeleDB-ADB is about 5% better than that of a well-known database (see figure below).

In a hybrid data warehouse (later called data warehouse) big data open platform CDAP project, TeleDB-ADB solved the complex analysis and high concurrency problems encountered by users in online data application. When faced with complex data analysis, the original system of the enterprise is difficult to give results in a reasonable time, so the business has to split a large table into several small tables, and then analyze the data, summarize and check the results, while TeleDB-ADB can produce correct results in minutes.

For example, in the business scenario of the enterprise's multi-table association query, 42 tables need to be joined, in which the main table contains 24 million pieces of data, the data volume of the associated table is between 10 million and 40 million, and the analysis requirements need to be associated by left join. TeleDB-ADB obtains the correct results in about 2 minutes to help users quickly solve all kinds of complex analysis needs in the business.

Automatic allocation of resources to reduce cost and increase efficiency

According to the best practices of multiple users, the investment of TeleDB-ADB is about 1 beat 3 less than the traditional warehouse, giving users the ultimate TCO (total cost of ownership). This is because traditional warehouses need to prepare resources based on computing and storage peaks, while the actual analysis needs of enterprises are volatile (as shown by the gray line in the following figure), which means that at some point, the computing and storage resources of these warehouses may be wasted. TeleDB-ADB adopts an excellent architecture of complete decoupling of computing and storage. While object-based storage provides almost unlimited scalability and high flexibility, it can automatically expand and scale computing resources, release computing resources when the business does not need them, and add them when needed, and its TCO fluctuates in different periods (as shown in the red module below).

In addition, TeleDB-ADB 's metadata system ensures that data is globally visible, thus supporting multiple computing requirements to use the same data for calculation without the need for additional copies. For example, in the scenarios that need to store and use a large amount of historical data, such as big data analysis, multi-year history analysis and massive log analysis, the metadata system greatly improves the utilization of storage resources and the efficiency of data governance.

In addition, TeleDB-ADB is based on the SQL:2003 standard, provides complete transaction capabilities, is compatible with PostgreSQL and Greenplum protocol syntax, and supports standard JDBC and ODBC interfaces. This means that TeleDB-ADB is compatible with most mature BI or ETL tools on the market, allowing enterprises to migrate to TeleDB-ADB almost seamlessly.

Unlimited flexibility, easy to cope with huge amounts of data

In the face of the needs of enterprises to open up the departmental data wall and build a unified data center, TeleDB-ADB can provide access to many departments and branch offices at the same time, allowing different application requirements to share a data warehouse system. Compared with the traditional data warehouse, TeleDB-ADB has better concurrency, scalability, and higher capacity.

In comparison, although the traditional data warehouse is flexible and scalable, the computing and storage in its architecture are tightly coupled. When capacity expansion involves data redistribution, data warehouses usually store data at the TB or PB level, and it takes at least several hours or more to complete a data redistribution. Some traditional data warehouses can not write data normally during the redistribution period, which will affect the normal operation of some businesses. This also means that most traditional warehouses still need peak-based reserve resources.

On the other hand, TeleDB-ADB can expand the capacity of computing and storage resources separately, so that the database can still provide services in the process of expansion. It can expand according to the deficiency of storage and computing resources, and ensure that they do not affect each other while realizing minute-level expansion. When there are no computing tasks, TeleDB-ADB can also release computing resources without affecting data persistence and accessibility, which greatly improves the efficiency of the company's use of computing resources.

Security first, protect the data treasure.

As an analytical database, TeleDB-ADB attaches great importance to the building of data security capability, has passed the three-level filing and secret review inspection of national security, and provides multiple security features such as multiple identity authentication, mandatory access control, data encryption, security audit and other security features, as well as rights accurate to row and column levels, effectively ensuring the security and privacy of users.

TeleDB-ADB has high availability and disaster recovery backup capabilities, and supports cross-region, multi-location and multi-center deployment architectures. Even in extreme cases, it can ensure the integrity and security of user data by relying on fault self-healing and strong data consistency.

Generally speaking, TeleDB-ADB relies on advanced architecture, has strong data processing capacity, flexible scalability, reliable security and perfect user service system, and has accumulated rich landing experience in practice. For enterprises that want to improve the efficiency of data processing, reduce costs and gain more business insight, through TeleDB-ADB, enterprises can better tap the value of data and provide strong support for business decisions.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

IT Information

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report