Skip to main content

Replacing Cloudera Manager: Apache Ambari and Ilum Cluster Ops

A Classic migration removes Cloudera Manager. The migrated cluster is managed by Apache Ambari (Ilum build), operated directly or through Ilum Cluster Ops. Ambari takes over the role Cloudera Manager had: it holds the desired configuration of every service, starts and stops daemons, restarts services in a rolling fashion, runs service checks, and raises alerts.

The takeover is phase 9 of the Classic pipeline. It runs inside the migration window and is the step that starts the migrated services on the default, version-aligned target. The services, their configuration, and their security settings are carried from the source; nothing is rebuilt from defaults.

How the Takeover Works​

The management plane is set up and adopted in three commands:

  1. bifrost classic stage-ambari installs and sets up the Ambari server from the delivered bundle.
  2. bifrost classic validate-ambari (optional) checks that every Cloudera Manager role maps to a component of the target stack, and lets Ambari validate the adoption blueprint without creating a cluster.
  3. bifrost classic ambari-takeover adopts the cluster. Without --execute it checks the adoption blueprint and creates nothing.
export BIFROST_AMBARI_PASSWORD='...'
bifrost classic stage-ambari --cluster prod01 --from /srv/bifrost/ambari-bundle
bifrost classic validate-ambari --cluster prod01 --ambari-host ambari.example.internal
bifrost classic ambari-takeover --cluster prod01 \
  --ambari-host ambari.example.internal \
  --from /srv/bifrost/ambari-bundle \
  --execute

With --execute, the takeover runs these steps in order:

  1. The configuration of every node is saved as a pre-adoption snapshot, which a rollback restores.
  2. The Ambari agent is installed on every node from the local package cache and registered with the server.
  3. The cluster is created in Ambari with an install-only request, so that Ambari starts nothing on its own.
  4. The estate's own keytabs are placed, its TLS material is staged, and Kerberos is enabled for the cluster.
  5. Every service is started explicitly, in dependency order.
  6. Alert targets and Cloudera Manager replication schedules are carried; the schedules are imported in a paused state.
  7. Ambari automatic restart is enabled for every adopted daemon.
  8. The adopted configuration is certified as the baseline for later drift detection.

An interrupted takeover is resumed with --resume-from, which verifies the steps before the named one instead of trusting that they ran. ambari-takeover --un-adopt --execute removes the Ambari cluster again. The full inverse, which also restores the pre-adoption configuration and removes Ambari, is bifrost classic rollback --from-adopted. See Validation and rollback.

What Is Carried​

  • Configuration. Every service configuration is carried with its source values. Where Cloudera Manager role groups gave hosts different values, the differences become Ambari configuration groups.
  • Kerberos identities. The estate's own keytabs are copied to their places; no principal is created and no key version changes. See Security and encryption.
  • TLS material. The estate's own keystores and truststores are reused, never reissued.
  • Ranger plugins. The plugins of every service are installed and connected to Ranger Admin, and policy content is compared before and after the cutover.
  • Atlas hooks. The Hive, HBase, Sqoop, Impala, and Spark hooks are carried, so lineage keeps flowing to Atlas.
  • Replication schedules. HDFS schedules (from a peer cluster, between clusters of the same Cloudera Manager, and to Amazon S3) and Hive schedules are imported into the replication schedules of Apache Ambari (Ilum build) in a paused state. After validation, bifrost classic replication-activate runs a dry run of each schedule and resumes the schedules that read their source. See Replication.
  • Console login. When Cloudera Manager authenticated its console users through LDAP, the Ambari console login is configured from the same directory settings.

Ambari Server Setup​

SettingBehaviour
DatabasePostgreSQL by default: a separate ambari database on the Hive Metastore PostgreSQL instance when the metastore runs PostgreSQL, otherwise a dedicated local PostgreSQL. Oracle is supported as well.
Port8085 by default (8080 is used by the ZooKeeper admin server); 8443 for HTTPS.
Cloudera Manager endpointWith ambari_cm_endpoint_takeover: true, Ambari serves on the Cloudera Manager ports 7180 and 7183, so bookmarks and integrations keep working. The Ambari server must then run on the former Cloudera Manager host.
HTTPSAmbari serves HTTPS with the estate's own certificate when the source cluster used TLS.
Agent channelTwo-way TLS between the Ambari server and its agents is available and off by default.
SizingThe server heap and connection pools are sized from the number of cluster nodes.
InstallationAll packages are installed from the delivered bundle; no network access is needed.

Automatic Restart​

The takeover enables Ambari automatic restart for every adopted daemon. Unlike Cloudera Manager, Ambari also restarts a daemon that stopped unexpectedly while its host stayed up. Daemons are therefore stopped through Ambari or Ilum Cluster Ops, not by stopping the process on the host, because a process stopped outside the management plane is started again.

Monitoring and Alerts​

  • Ambari alerts. The stack alert definitions run from the first start of the adopted cluster.
  • Alert targets. Email and SNMP targets are derived from the Cloudera Manager alert configuration: the same mail relay, sender, and recipients. They are carried but not bound until the cluster owner arms them with ambari_alert_targets_arm: true, so that no alert reaches a distribution list before the cutover is accepted.
  • Prometheus and Grafana. A Prometheus and Grafana service is available on request (bifrost_adopt_monitoring: true). Prometheus listens on port 9091 by default, because 9090 is used by the HBase Thrift server. The JMX exporter ports that Cloudera Manager configured are carried.

Configuration Plane​

The takeover records the configuration Ambari holds as the certified baseline. validate-post names every later change made outside Bifrost by service, configuration type, and key, for example a value rewritten by the Ambari stack advisor in a configuration wizard. bifrost classic restore-config-plane puts a changed service back to its certified version. See Day-2 operations: configuration plane control.

Ambari Database Backup​

  • Scheduled backup. With ambari_db_backup_enabled: true, Bifrost installs a daily backup of a PostgreSQL Ambari database, kept for 14 days. The same backup can be run by hand before an upgrade.
  • Oracle. An Oracle Ambari database is backed up by the database team.
  • Server recovery. The Ambari server runs as a single instance. To recover it on another host, restore the database and point the agents to the new server; a DNS alias for the server name keeps the agent configuration unchanged.

Changes for Operators​

Cloudera ManagerAfter the migration
Console users and rolesAmbari or Ilum Cluster Ops accounts. Ranger administration roles are carried with Ranger.
Automation that calls the Cloudera Manager APIRe-pointed to the Ambari REST API, or served by the Cloudera Manager API compatibility layer of Apache Ambari (Ilum build).
YARN Queue ManagerQueues are managed in the YARN Queue Manager view of Apache Ambari (Ilum build); queue definitions and placement rules are carried.
Service portsUnchanged. Every port a source role listened on is checked on the same host after the cutover.

Next Steps​

Last updated on