Package org.apache.hadoop.hbase.mapreduce – Provides HBase MapReduce Input\/OutputFormats, a table indexing MapReduce job, and utility methods.<\/p>\n
VisibilityExpressionResolver \u00a0 \u00a0– Interface to convert visibility expressions into Tags for storing along with Cells in HFiles.<\/p>\n
The following is an example of using HBase as a MapReduce source in a read-only manner:<\/p>\n
Configuration config = HBaseConfiguration.create();<\/p>\n
config.set(\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ speculative<\/p>\n
\u201cmapred.map.tasks.speculative.execution\u201d, \/\/ execution will<\/p>\n
\u201cfalse\u201d);\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ decrease performance<\/p>\n
\/\/ or damage the data<\/p>\n
Job job = new Job(config, \u201cExampleRead\u201d);<\/p>\n
job.setJarByClass(MyReadJob.class); \/\/ class that contains mapper<\/p>\n
Scan scan = new Scan();<\/p>\n
scan.setCaching(500);\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ 1 is the default in Scan,<\/p>\n
\/\/ which will be bad for MapReduce jobs<\/p>\n
scan.setCacheBlocks(false);\u00a0\u00a0 \/\/ don\u2019t set to true for MR jobs<\/p>\n
\/\/ set other scan attrs<\/p>\n
…<\/p>\n
TableMapReduceUtil.initTableMapperJob(<\/p>\n
tableName,\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ input HBase table name<\/p>\n
scan,\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ Scan instance to control CF and attribute selection<\/p>\n
MyMapper.class,\u00a0\u00a0 \/\/ mapper<\/p>\n
null,\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ mapper output key<\/p>\n
null,\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/\/ mapper output value<\/p>\n
job);<\/p>\n
job.setOutputFormatClass(NullOutputFormat.class); \/\/ because we<\/p>\n
\/\/ aren\u2019t emitting anything from mapper<\/p>\n
boolean b = job.waitForCompletion(true);<\/p>\n
if (!b) {<\/p>\n
throw new IOException(\u201cerror with job!\u201d);<\/p>\n
}<\/p>\n
The mapper instance would extend TableMapper, too, like this:<\/p>\n
public static class MyMapper extends TableMapper<Text, Text> {<\/p>\n
public void map(ImmutableBytesWritable row, Result value, Context context)<\/p>\n
throws InterruptedException, IOException {<\/p>\n
\/\/ process data for the row from the Result instance.<\/p>\n
}<\/p>\n
}<\/p>\n
When TableInputFormat is used (set by default with TableMapReduceUtil. initTableMapperJob(…)) to read an HBase table for input to a MapReduce job, its splitter will make a map task for each region of the table. Thus, if 100 regions are in the table, there will be 100 map tasks for the job, regardless of how many column families are selected in the Scan. To implement a different behavior (custom splitters), see the method getSplits in TableInputFormatBase (either override in custom-splitter class or use as example).<\/p>\n
Writing to HBase<\/p>\n
The following is an example of using HBase both as a source and as a sink with MapReduce:<\/p>\n
Configuration config = …; \/\/ configuring reading<\/p>\n
Job job = …; \/\/ from HBase table<\/p>\n
Scan scan = …; \/\/ is the same as in<\/p>\n
TableMapReduceUtil \/\/ read-only example<\/p>\n
.initTableMapperJob(…); \/\/ above<\/p>\n
TableMapReduceUtil.initTableReducerJob(<\/p>\n
targetTable, \/\/ output table<\/p>\n
MyTableReducer.class, \/\/ reducer class<\/p>\n
job);<\/p>\n
job.setNumReduceTasks(1); \/\/ at least one, adjust as required<\/p>\n
boolean b = job.waitForCompletion(true);<\/p>\n
And the reducer instance would extend TableReducer, as shown here:<\/p>\n
public static class MyTableReducer extends TableReducer<Text, IntWritable,<\/p>\n
ImmutableBytesWritable> {<\/p>\n
public void reduce(Text key, Iterable<\/p>\n
values, Context context)<\/p>\n
throws IOException, InterruptedException {<\/p>\n
…<\/p>\n
Put put = …; \/\/ data to be written<\/p>\n
context.write(null, put);<\/p>\n
…<\/p>\n
}<\/p>\n
}<\/p>\n
http:\/\/www.vskills.in\/certification\/Certified-HBase-Professional<\/a><\/p>\n Package org.apache.hadoop.hbase.mapreduce – Provides HBase MapReduce Input\/OutputFormats, a table indexing MapReduce job, and utility methods. Interface VisibilityExpressionResolver \u00a0 \u00a0– Interface to convert visibility expressions into Tags for storing along with Cells in HFiles. Class CellCounter – A job with a a map and reduce phase to count cells in a table. CellCreator – Facade to…<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[69],"tags":[],"class_list":["post-20786","page","type-page","status-publish","hentry","category-hbase"],"acf":[],"yoast_head":"\nGo To- Certified HBase Professional Tutorial<\/strong><\/a><\/h5>\n<\/div>\n","protected":false},"excerpt":{"rendered":"