Skip to main content

Distcp

Command: 

hadoop distcp -libjars $jar1 --files $file1 -Dgoogle.cloud.auth.service.account.json.keyfile=SERVICE_ACCOUNT.json -Dfs.AbstractFileSystem.gs.impl=com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS -m 1 hdfs://${hadoop_loc}  gs://${gcs_loc}

Comments