Hadoop: Setting up single node Cluster on Linux desktop

Following are the helpful links for setting up Hadoop on Linux

Successfully setup the Hadoop cluster on LINUX desktop.

1. http://www.bogotobogo.com/Hadoop/BigData_hadoop_Install_on_ubuntu_single_node_cluster.php
    http://qursaan-howto.blogspot.com/2014/12/how-to-install-hadoop-as-single-node-on.html

2. Editing read only file in VI editor 
    http://www.geekyboy.com/archives/629

command to save the read only files

:w !sudo tee %
O

3. VI editor commands for editing configuration files
    http://www.cs.rit.edu/~cslab/vi.html
    http://www.cs.colostate.edu/helpdocs/vi.html

Hadoop: Install Rstudio Server on Horton Works Hadoop Sandbox

Steps to Install RSTUDIO in HortonWorks Sandbox

1.  Login to HortonWorks sandbox with root user
2.  Install RStudio Server
3.   Create a different user (sandbox)
4.  Add port forward in VirtualBox
5.  Restart Hortonwors Hadoop Sandbox
6. Login to Sandbox other than root user (Use user created in step 3)
7.  Open the RStudio in web browser.
8.  Open Google Chrome and type htt127.0.0.1:8787

Login to HortonWorks Sandbox

·         Start the Sandbox

·         Click ALT+F5

·         Login with user root and pwd hadoop  

INSTALL RSTUDIO SERVER

Run the following commands in the sandbox console

wget http://download2.rstudio.org/rstudio-server-0.97.332-x86_64.rpm

sudo yum install --nogpgcheck rstudio-server-0.97.332-x86_64.rpm

CREATE ANOTHER USER
useradd sandbox
passwd sandbox

This is because you can not use root to log into RStudio Server. Here I use sandbox as username, but you can choose whatever you want.

ADDING PORT FORWARD IN VIRTUALBOX

1.       Go To Devices à Network à Network Settings

2.       Click on Advanced à Port Forwarding (If Advanced is expanded you can see Port Forwarding)

3.       Click on Add button

4.       Add the RStudio port forwarding entries

5.       You can browse the RSTUDIO from the browser using http://127.0.0.1:8787

6.  You need to use Google Chrome browser

Hadoop: Adding PORT forward to access Rstudio server from Hortonworks Hadoop Sandbox in VirtualBox

After installing RStudio Server on Hortonworks Hadoop Sandbox, to access RStudio from the browser (Google Chrome), need to set up port forwarding in VirtualBox.

Use the following steps for setting up port forwarding.

ADDING PORT FORWARD IN VIRTUALBOX

1. Go To Devices à Network à Network Settings
2.  Click on Advanced à Port Forwarding (If Advanced is expanded you can see Port Forwarding)
3. Click on Add button
4. Add the RStudio port forwarding entries
5. You can browse the RSTUDIO from the browser using http://127.0.0.1:8787
6. You need to use Google Chrome browser.

Network Settings

Port Forwarding

Adding Port for forwarding

Hadoop: Exploring Hadoop Sandbox files from Windows System

Problem: Members who are working on Windows OS with Hadoop sandbox or Hadoop not able to view or move the files from Windows OS to Hadoop Sandbox.

Solution:
1.You can use HUE interface to upload files from windows to Hadoop.
2. You can install HDFS Explorer on windows and explorer the files. This will help you to identify where the files are present in Hadoop sandbox.
3. you can use the following Hadoop commands
Hadoop fs -ls /<<folderpath>>

Install HDFS Explorer from the following link.

http://bigdata.red-gate.com/

http://hadoop.drawbackz.com/stack/12821/how-to-transfer-files-from-windows-to-hortonworks-sandbox.html

http://bigdata.red-gate.com/connecting-to-sandboxes.html

How to retreive random number of rows from a table

USE THE TABLESAMPLE KEYWORD AFTER THE TABLE NAME.

Following queries retrieve random number of rows each time

SELECT * FROM TABLE_NAME TABLESAMPLE(100 ROWS)

SELECT * FROM TABLE_NAME TABLESAMPLE(10 PERCENT)