SparkR gapply mess - 2017-05-12 08:56:31

Hello, Do not assume anything. Never. Ever. Specially with SparkR (Apache Spark 2.1.0). When using the gapply function, maybe you want to return the key to mark the results in a function as follows: countRows <- function(key, values) { df <- data.frame(key=key, nvalues=nrow(values)) return(df) } count <- gapplyCollect(data, "keyAttribute", countRows) countRows <- function(key, values) { df <- data.frame(key=key, nvalues=nrow(values)) return(df) } count <- gapplyCollect(data, "keyAttribute", countRows) SURPRISE. You can’t. You should get this error: