Big data DMP Portrait system
Content introduction
I. objectives
1. Master the skills of developing portrait tags.
2. Master data mining skills
3. Understand the architecture and development of portraits and DMP systems in the industry.
4. Big data landed in combination with the business scene
System development requirements
Technical points involved: spark, elasticsearch, hadoop, hive, LR GBDT and other machine learning algorithm development tools: idea, eclipse development environment: spark2.2, hadoop2.7, hive1.2, hbase, redis development languages: scala, java, python, shell, sql
III. Course catalogue
Course list
1. Overview of user portraits
What is the user portrait, why should the user portrait, the scene of the portrait application industry, facebook, Alibaba (dharma plate), Tencent (Tencent Ad solutions) analysis to create their own internal Dama plate, the basic functions are the same as the Dama plate
2. Collation of portrait indicators
2-1. Basic properties. People's basic attribute tags, including region, age, gender and so on.
2-2. Interest preference. This part is the existing orientation ability of the delivery end, and later, you can plan more detailed search options based on babies, stores or industries, and orientation functions of specific interests.
2-3. Behavior trajectory. More detailed behavior based on interest preference (including browsing, clicking, closing, collection, repurchase, etc.), and behavior intersection in different time periods (including 1-day, 7-day, 30-day behavior).
2-4. Spending power. Platform-based payment transactions, shopping behavior, transaction volume calculation of high, medium and low, and category of high consumption preference.
2-5. Good friends. Based on the relationship chain data of the platform, it is recommended to prefer the friends of the baby, store and industry.
2-6. Custom crowd. Support the upload of custom crowd packages and the size of lookalike expansion packages.
3. Construction and development of portrait label system.
3-1) basic attributes: region, age, gender, educational background, occupation
3-2) interest preference brand, store, first-tier category, scene, industry
3-3) Grade development of consumption capacity
3-4) the characteristic population is divided into some specific groups, high active, low active, car family, nanny family.
3-5) LBS attribute permanent residence
3-6) user track trading, browsing, collection, etc.
IV. Architecture of Portrait system
Function: portrait multi-dimensional analysis, portrait index drill-down analysis, release effect tracking analysis technology: construction of portrait calculation based on es, spark, hadoop, and data storage and calculation module: user crowd package (intersection and union), tracking analysis, crowd portrait, crowd comparison module development
Fifth, the application case of portrait system
Accurate marketing of users
User commodity recommendation
Big data's interview skills
Common interview questions and answers for hadoop, hive and spark
