Must Have Skills :
| 5+ years of hands-on experience with Apache NiFi, including developing, managing, and optimizing complex data flows in production environments. • Proven experience with Cloudera NiFi (CDP Data Flow) in enterprise environments, including integration with Cloudera Manager. • Experience migrating NiFi flows across major version upgrades with strong understanding of backward compatibility • Strong proficiency in Groovy scripting, used for ExecuteScript and InvokeScriptedProcessor processors. • Solid understanding of SSH and SFTP protocols, including authentication schemes (key-based, password), session negotiation, and file permissions handling in NiFi processors (e.g., ListSFTP, FetchSFTP, PutSFTP). |
Good to Have Skills :
| Experience working in regulated or secure environments, with strict data handling and audit requirements. • Familiarity with DevOps workflows, including version-controlled flow templates (JSON/XML), CI/CD integration for NiFi Registry, and automated deployment strategies. • Strong written and verbal communication skills, with ability to document flows and onboard other engineers.
|
Roles and Responsibilities :
| Good grasp of data encryption mechanisms, key management, and secure flowfile handling using processors like EncryptContent. • Experience integrating NiFi with MongoDB, including reading/writing documents via processors like GetMongo, PutMongo, and QueryMongo. • Experience working with Apache Kafka, including producing and consuming from Kafka topics using NiFi (PublishKafka, ConsumeKafka), and handling schema evolution with Confluent Schema Registry. • Strong knowledge of Red Hat Enterprise Linux (RHEL) environments, including systemd services, filesystem permissions, log rotation, and resource tuning for JVM-based applications like NiFi. NiFi-Specific Technical Requirements: • In-depth knowledge of NiFi flow design principles, including proper use of queues, back pressure, prioritizers, and connection tuning. • Mastery of controller services, including SSLContextService, DBCPConnectionPool, and RecordReader/RecordWriter services. • Experience with Record-based processing using Avro, JSON, CSV schemas and Record processors like ConvertRecord, QueryRecord, and LookupRecord. • Ability to debug and optimize NiFi flows using Data Provenance, bulletins, and log analysis. • Familiarity with custom processor development in Java/Groovy (optional but preferred). • Experience setting up secure NiFi clusters, configuring user authentication (LDAP, OIDC), TLS certificates, and access policies. • Proficiency in parameter contexts, variable registry, and flow versioning using NiFi Registry. • Understanding of Zero-Master clustering model, node coordination, and site-to-site protocol. • Experience deploying and monitoring NiFi in high-availability, production-grade environments, including using Prometheus/Grafana or Cloudera Manager for metrics and alerting. |