Sahara Plugins¶
As you continue to test things out, put information you learn here. Like default passwords, etc. and other things that aren’t immediately intuitive to even the advanced Hadoop user.
Validation means the minimal configuration under which Sahara will launch a cluster. There are other considerations for node processes other than what is listed here. Also note that Sahara will again validate cluster during scaling.
Vanilla¶
Description - The Apache Vanilla plugin provides the ability to launch upstream Vanilla Apache Hadoop cluster.
Version 2.7.1
Node processes
- namenode
- datanode
- secondarynamenode
- resourcemanager
- nodemanager
- historyserver
- oozie
- hiveserver
Validation
- Cluster must contain exactly one namenode
- Cluster can contain at most one resourcemanager
- Cluster can contain at most one secondary namenode
- Cluster can contain at most one historyserver
- Cluster can contain at most one oozie and this process is also required for EDP
- Cluster can’t contain oozie without resourcemanager and without historyserver
- Cluster can’t have nodemanager nodes if it doesn’t have resourcemanager
- Cluster can have at most one hiveserver node.
Hortonworks (”Ambari plugin”)¶
Description - The Ambari plugin provides a way to provision clusters with Hortonworks Data Platform using Ambari as the orchestrator (blueprints).
Version 2.2 on Liberty, 2.3 on Mitaka
Node processes
- Flume
- DataNode
- NameNode
- SecondaryNameNode
- Sqoop
- Ambari
- Ranger Admin
- Ranger Usersync
- ZooKeeper
- Hive Metastore
- HiveServer
- Kafka Broker
- Slider
- YARN Timeline Server
- MapReduce History Server
- NodeManager
- ResourceManager
- DRPC Server
- Nimbus
- Storm UI Server
- Storm Supervisor
- Oozie, Falcon Server
- HBase Master
- HBase RegionServer
- Knox Gateway
- Spark History Server
Validation
- Ensure the existence of Ambari Server process in the cluster
- Ensure the existence of a NameNode, Zookeeper, ResourceManager(s), HistoryServer and App TimeLine Server in the cluster
MapR¶
Description - The MapR Distribution provides a full Hadoop stack that includes the MapR File System (MapR-FS), MapReduce, a complete Hadoop ecosystem, and the MapR Control System user interface
Version 5.0.0 on Liberty, 5.1.0 on Mitaka
Node processes
- Flume
- Hue
- Hue Livy
- ZooKeeper
- Webserver
- Metrics
- HTTPFS
- HiveMetastore
- HiveServer2
- Pig
- Mahout
- ResourceManager
- NodeManager
- HistoryServer
- Drill
- Oozie
- HBase-Master
- HBase-RegionServer
- HBase-Thrift
- CLDB
- FileServer
- NFS
- Impala-Catalog
- Impala-Server
- Impala-StatestoreSqoop2-Client
- Sqoop2-Server
Validation
in-depth,
follow validation rules for MapReduce v2 when applicable
Cloudera¶
Description - The Cloudera plugin provides the ability to launch the Cloudera distribution of Apache Hadoop (CDH) with Cloudera Manager management console
Version 5.4.0 on Liberty, 5.5.0 on Mitaka
Node processes
- IMPALA_CATALOGSERVER
- ZOOKEEPER_SERVER
- IMPALA_STATESTORE
- KMS, FLUME_AGENT
- HUE_SERVER
- CLOUDERA_MANAGER
- YARN_JOBHISTORY
- IMPALAD
- HBASE_REGIONSERVER
- HBASE_MASTER
- OOZIE_SERVER
- HDFS_DATANODE
- SENTRY_SERVER
- YARN_RESOURCEMANAGER
- HDFS_SECONDARYNAMENODE
- HIVE_METASTORE
- HDFS_NAMENODE
- YARN_STANDBYRM
- SOLR_SERVER
- YARN_NODEMANAGER
- KEY_VALUE_STORE_INDEXER
- HDFS_JOURNALNODE
- SPARK_YARN_HISTORY_SERVER
- HIVE_SERVER2
- SQOOP_SERVER
- HIVE_WEBHCAT
Validation in-depth
