Shulou(Shulou.com)06/03 Report--

Implicit conversion in RDD source code

Def distinct (numPartitions: Int) (implicit ord: Ordering [T] = null): RDD [T] = withScope {

Map (x = > (x, null)). ReduceByKey ((x, y) = > x, numPartitions). Map (_. _ 1)

}

Implicit parameters are used, the runtime does not need to provide the parameter ord manually, and the runtime injects the parameters according to the context

Def groupBy [K] (f: t = > K) (implicit kt: ClassTag [K]): RDD [(K, Iterable [T])] = withScope {

GroupBy [K] (f, defaultPartitioner (this))

}

Def groupBy [K] (

F: t = > K

NumPartitions: Int) (implicit kt: ClassTag [K]): RDD [(K, Iterable [T])] = withScope {

GroupBy (f, new HashPartitioner (numPartitions))

}

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.