System architect (java) and big data architect

Architect is not a professional job, but a kind of ability, and architects are divided into many kinds, and architects in different fields are different. For example, there is no comparison between Internet architects and Internet of things architects.

What does the architecture consider?

1. Considering what the system can and cannot do is what is often referred to as the system boundary.

two。 Determine the relationship between modules within the architecture and between modules, and what is the relationship between module and external

3. Determine non-functional requirements, scalability, availability, maintainability, and security of the architecture

4. In the future, the architecture should be able to guide developers to design and evolve according to the architecture idea, and ensure that the developed things are consistent with the architecture planning.

Java system architect

System technical selection and feasibility evaluation distributed technical framework control in-depth mining and analysis of business requirements, design of key modules, scheme review and technical support, technical difficulties, how to implement distributed system technology, write technical solutions and system design documents, guide developers to complete coding work for product performance and architecture optimization, ensure excellent architecture quality of the system abstract design capabilities Clear thinking, good at thinking, able to analyze and solve problems independently.

You may need to master the following

Familiar with * nix, proficient in using shell/python etc. Solid basic knowledge of java, familiar with IO, multithreading and concurrent programming, NoSQL,Netty,kafka and other middleware technologies have a solid understanding of JVM principles, in-depth understanding of server-side design and programming, familiar with common data structures, familiar with common design patterns, in-depth understanding of distributed algorithms and mainstream distributed systems Familiar with mainstream message queue and its working principle (sort/index/store/smooth/duplicate removal). Rich experience in distributed, high concurrency, high load, high availability system design, familiar with distributed cache, message mechanism, etc.

Big data architect

Formulate data architecture specifications for the design and development of big data platform, write core code to upgrade and optimize data infrastructure and data processing system, solve technical problems, continuously improve core system performance, ensure system safe, efficient and stable operation of large-scale data real-time, technology containerization, private cloud implementation skills scheme, data model standardization, etc., track the technical trends of big data related fields. The development of competitive products and technical trends proficient or proficient in big data ecological technology and products: Hive,Storm,Flink,Spark,ELK,Kafka,Zookeeper,Yarn,presto, in-depth understanding of the underlying principles of spark distributed computing, practical experience in performance optimization and stability of complex systems, proficient in data warehouse design, deep understanding of MR operating principles and mechanisms, can optimize task execution efficiency Familiar with open source data exchange tools: sqoop,Streamset,kettle,datax etc. Be good at capturing business needs, plan architecture design according to needs, and give corresponding solutions and methods. Additional items: data prediction, early warning, artificial intelligence, image recognition, voice intelligence, Internet of things data collection, marking, etc. Linear computation, neural network, knowledge graph.

The similarities and differences between the two:

Different: java system architects focus on distributed, high availability, concurrency, fault tolerance, etc. Proficient in the system architecture or tool set to support the business

Big data architect focuses on artificial intelligence, business intelligence, data mining, offline, real-time and other efficient and highly available structured data, semi-structured data. Familiar with the hadoop ecosystem, using a variety of model component sets to solve problems.

Same: reduce cost and increase efficiency, improve efficiency and create value.