Skip to main content

Hadoop Cluster Architecture and Core Components



  • Cluster means Many Computers are worked together as one system.Hadoop Cluster means Computer Cluster used at Hadoop.
  • Hadoop Cluster Mainly designed for storing large amount of unstructed data in Distributed file systems.
  • It referred as “Shared Nothing” Systems and shared data between nodes.
  • Hadoop Clusters are Arranged in racks and it having three nodes which is worker node,master node and Client nodes.
Hadoop Cluster Architecture:
  • Hadoop Cluster Having 110 racks and that racks having slave machines.
  • One Rack Switch are placed on top of the Each rack
  • Slave Machines are connected as cables which connected on rack switch.
  • Rack Switch contains 80 Ports
  • NameNode and Job tracker of hadoop cluster are refered as slaves only.
Hadoop Cluster has three core Components
  • Client
  • Master
  • Slave
Client:
Main purpose of Client are submitted the Mapreduce jobs
Client should be describes how to all data are processed
Finally retrives all data after completion of job submission
Master:
Master having 3 nodes
1.NameNode
2.Secondary Node
3.Job tracker
In Hadoop Cluster NameNode stores file metadata only .NameNode is the health of datanode and it access datanode data only.
NameNode Tracking all information from files such as
  • which file saved in which cluster
  • Access time of file
  • Which user access a file on current time
Purpose of Secondary name node is contact namenode at each one hour and collects the file metadata from namenode and it copy file meta data in namenode.
After collection of file meta data secondary namenode should be clean the file metadata folder.
Finally send new file metadata to namenode.
Job tracker mainly used for submit mapreduce jobs and manges mapreduce jobs also.
Job tracker manages data node and tracking available resoures and already running task.
Slave:
Slaves are responsible for processing data and store the data
It runs on data node and task tracker
It acts as data node and namenode of hadoop cluster
Follow us for more Interesting Topics in Hadoop

Comments

Post a Comment

Popular posts from this blog

Image
CAP Theorem in Hadoop What is CAP Theorem? CAP theorem is designed for distributed file systems(collection of interconnected nodes).CAP Theorem also known as Brewer’s theorem and used to distributed consistency.It contains follwing three technical terms for distributed systems. C  – Consistency A  – Availability P  – Partition Tolerance Consistency: When you read data it will give same data how many times read and server send response each and every request but systems always consistent when read data.(all node having same data) Availability: It means all requests give response and no error accured in this systems. Partition Tolerance: All functions run all time when more nodes not responsive and commnication break between two nodes Distributed systems statisfy any two terms only and not satisfy three terms Selecting Two options in CAP Theorem: CP – Consistency/Partition Tolerance: It wait for response form partioned nodes and that data are timeo
Read more
Image
How to Get Hadoop Developer Jobs Who is Hadoop Developer? Hadoop Developer  role is similar to Software Developer.Responsible of hadoop developer is programming and develop the hadoop applications and all components of hadoop ecosystem.Here we discuss about main roles and responsibilities to become a hadoop Developer. Click Here to Re ad Full Article
Read more