What are the common skills of Pandas
This article mainly shows you "what are the common skills of Pandas", the content is simple and clear, and I hope it can help you solve your doubts. Let the editor lead you to study and learn this article "what are the common skills of Pandas?"
1. Calculate the missing rate of variables
Df=pd.read_csv ('titanic_train.csv') def missing_cal (df): "df: dataset return: missing rate of each variable"missing_series = df.isnull (). Sum () / df.shape [0] missing_df = pd.DataFrame (missing_series) .reset_index () missing_df = missing_df.rename (columns= {' index':'col') Missing_df = missing_df.sort_values ('missing_pct',ascending=False) .reset_index (drop=True) return missing_df missing_cal (df)
If you need to calculate the missing rate distribution of the sample, just add the parameter axis=1.
two。 The method of getting the row where the maximum value in the group lies
It is divided into two groups: repeated values and non-repetitive values.
The case where there is no duplicate value.
Df = pd.DataFrame ({'Sp': [' axiaoyunzhuo]], 'Mt': [' s 1','s 1','s 2'], 'Value': [1, 2, 3, 4, 5, 6],' Count': [3, 2, 5, 10, 10]}) dfdf.iloc [df.groupby (['Mt'])) .apply (lambda x: X [' Count']. Idxmax ()]]
First group according to the Mt column, then use the idxmax function to fetch the column where the maximum value of Count is located in the data box after the grouping, and then use the iloc location index to fetch the row.
The case of having duplicate values.
Df ["rank"] = df.groupby ("ID") ["score"] .rank (method= "min", ascending=False) .astype (np.int64) df [df ["rank"] = = 1] [["ID", "class"]]
After grouping the ID, the rank function is applied to the score. If the score is the same, the same ranking will be given, and then the data with rank 1 will be taken out.
3. Multiple columns are merged into one row
Df = pd.DataFrame ({'id_part':],' pred': [0.1, 0.2, 0.4], 'pred_class':,' women','man','cat','dog': ['d1, women','man','cat','dog', d3, d1]})
Df.groupby (['id_part':']) .agg ({'pred_class': [', '.join],' pred': lambda x: list (x), 'id_part':' first'}) .reset_index ()
4. Delete the line that contains a specific string
Df = pd.DataFrame ({'aqiang: [1Jing 2pas 3je 4], 'baked: [' s 1pm, 'exp_s2',' s 3pcDHI: [5pr 6pr 7pr 8], 'daddy: [3pr 2pr 5je 10]}) df [df [' b'] .str.coach ('exp')]
5. Intra-group sorting
Df = pd.DataFrame ([['Awardre 1], [' Amalagem 3], [''Achilles recorder 2], [' Barrie dagger 5], ['Barrie dagger 9]], columns = [' name','score']) df
This paper introduces two methods of efficient intra-group sorting.
Df.sort_values (['name','score'], ascending= [True,False]) df.groupby (' name') .apply (lambda x: x.sort_values ('score', ascending=False)) .reset_index (drop=True)
6. Select a specific type of column
Drinks = pd.read_csv ('data/drinks.csv') # Select all numeric columns drinks.select_dtypes (include= [' number']). Head () # Select all character columns drinks.select_dtypes (include= ['object']) .head () drinks.select_dtypes (include= [' number','object','category']) 'datetime']) .head () # excludes the specified data type drinks.select_dtypes (exclude= [' number']) .head () with the exclude keyword
7. Convert a string to a numeric value
Df = pd.DataFrame ({'column 1': ['1 'column 1'], 'column 2'], 'column 2: [' 4\ ^ 4\ {5\ ^ 5\]], 'column 3: [' 7\ ^ 7\]]) df df.astype ({'column 1: 1, column 1: 2, dtypes: 2].
Converting the third column in this way makes an error because the column contains an underscore for 0, which pandas cannot automatically determine. To solve this problem, you can use the to_numeric () function to process the third column and have pandas convert any invalid input to NaN.
Df = df.apply (pd.to_numeric, errors='coerce'). Fillna (0) df
8. Optimize DataFrame memory footprint
Method 1: read only the columns you really need, using the usecols parameter
Cols= ['beer_servings','continent'] small_drinks = pd.read_csv (' data/drinks.csv', usecols=cols)
Method 2: convert the object column containing type data to the Category data type, which is achieved by specifying the dtype parameter.
Dtypes = {'continent':'category'} smaller_drinks = pd.read_csv (' data/drinks.csv',usecols=cols, dtype=dtypes)
9. Filter DataFrame based on the largest category
Movies = pd.read_csv ('data/imdb_1000.csv') counts = movies.genre.value_counts () movies [movies.genre.isin (counts.nlargest (3) .index)] .head ()
10. Divide a string into multiple columns
Df = pd.DataFrame ({'name': ['Zhang San','Li Si', 'Wang Wu'], 'location': [Beijing-Dongcheng District', 'Shanghai-Huangpu District', 'Guangzhou-Baiyun District']}) df df. Name .str.split ('', expand=True)
11. Convert the list in Series to DataFrame
Df = pd.DataFrame ({'column 1x: [' axiomagemagem]], 'column 2p: [[10lemen 20], [20je 30], [30je 40]]}) dfdf_new = df. Column 2.apply (pd.Series) pd.concat ([df,df_new], axis='columns')
twelve。 Aggregate with multiple functions
Orders = pd.read_csv ('data/chipotle.tsv', sep='\ t') orders.groupby (' order_id'). Item_price.agg (['sum','count']). Head ()
13. Grouping polymerization
Import pandas as pd df = pd.DataFrame ({'key1': [' one', 'two',' one', 'two',' one'], 'data1':np.random.randn (5),' data2':np.random.randn (5)}) dffor name Group in df.groupby ('key1'): print (name) print (group) dict (list (df.groupby (' key1')
Group by dictionary or Series
People = pd.DataFrame (np.random.randn (5,5), columns= ['averse,' baked, 'crested,' dashed,'e'], index= ['Joe',' Steve', 'Wes',' Jim', 'Travis']) mapping = {' a'axiajuana, 'breadfruit, bluetooth,' cantilevered bluetooth, 'dumped bluetooth,' estranged columns= By_column = people.groupby (mapping, axis=1) by_column.sum ()
The above is all the content of this article "what are the Common skills of Pandas?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!
