HBase shell commands

describe ‘tablename’ : Displays the metadata of particular table Example : describe ’employee’  {NAME => ‘address’, BLOOMFILTER => ‘ROW’, VERSIONS => ‘5’, IN_MEMORY => ‘false’, KEEP_DELETED_CELLS => ‘FALSE’, DATA_BLOCK_ENCODING => ‘NONE’, TTL => ‘FOREVER’, COMPRESSION => ‘GZ’, MIN_VERSIONS => ‘0’, BLOCKCACHE => ‘true’, BLOCKSIZE => ‘65536’, REPLICATION_SCOPE => ‘0’} {NAME => ‘personal_info’, BLOOMFILTER => ‘ROW’, … More HBase shell commands

Data types of Spark Mlib – Part 2

Friends, so far we have gone through Basic types of spark mllib data type. Spark mllib also supports distributed matrices that includes Row Matrix , IndexedRowMatrix, CoordinateMatrix, BlockMatrix. A distributed matrix has long-typed row and column indices and double-typed values, stored distributively in one or more RDDs. It is very important to choose the right … More Data types of Spark Mlib – Part 2

Running scala oops files from command line- spark-shell

Writing a script in scala , but still want to follow object oriented programming as most of the programmer are from OOPs background due to java practice, can still execute the spark scripts using spark-shell in OOPs way. Here I am giving a simple example of CollectAsync() future method , also demonstrate how we can … More Running scala oops files from command line- spark-shell

Re-partitioning & partition in spark

  In Hadoop, partitioning a data allows processing of huge volume of data in parallel such that it takes minimum amount of time to process entire dataset. Apache spark decides partitioning based on different factors. Factor that decide default partitioning On hadoop split by HDFS cores. Filter or map function don’t change partitioning Number of … More Re-partitioning & partition in spark

Asynchronous processing in java

Asynchronous programming is very popular these days, primarily because of its ability to improve the overall throughput on a multi-core system. Asynchronous programming is a programming paradigm that facilitates fast and responsive user interfaces. The asynchronous programming model in Java provides a consistent programming model to write programs that support asynchrony. Asynchronous programming provides a … More Asynchronous processing in java