What Is Argo CD?
Argo CD is an open-source continuous delivery tool for Kubernetes. It automates the deployment of desired application states, defined declaratively, to a Kubernetes cluster. This tool is part of the larger Argo project, which includes a suite of tools for managing Kubernetes applications. Argo CD uses GitOps principles, meaning it views Git as the single source of truth for app declarations, ensuring consistent deployments.
Argo CD simplifies managing complex Kubernetes apps. By closely monitoring Git repositories, it ensures deployed images match the desired states in the repository. If any discrepancies are detected, Argo CD automatically synchronizes the cluster to match the declared state. This provides a mechanism for version control and application rollback, enhancing deployment reliability and security.
Argo CD Security Features
Authentication
Authentication in Argo CD is handled through JSON Web Tokens (JWTs), eliminating the need for traditional username and password bearer tokens. For the local admin user, JWT tokens are generated when a username and password are provided through the /api/v1/session
endpoint.
These tokens are issued and signed by the Argo CD API server and are configured to expire after 24 hours to limit their lifespan and minimize security risks. If the admin password is changed, all previously issued tokens are revoked immediately. Passwords are stored securely as bcrypt hashes within the argocd-secret
Kubernetes secret.
For Single Sign-On (SSO) users, JWTs are obtained by completing an OAuth2 login flow with a configured OIDC identity provider. Argo CD can be integrated with Dex, a popular OIDC provider bundled with Argo CD, or any other external OIDC provider. In this case, token management, including expiration and revocation, is handled by the identity provider itself. Tokens issued by Dex are also configured to expire after 24 hours.
Additionally, Argo CD allows the use of automation tokens, which are issued for specific projects. These tokens are generated via the /api/v1/projects/{project}/roles/{role}/token
API endpoint and have limited privileges, allowing them only to manage resources within the associated project. These tokens can be configured with expiration periods and revoked at any time by deleting the associated JWT reference ID.
Authorization
Authorization in Argo CD is governed by role-based access control (RBAC), which ensures that users can only perform actions for which they are explicitly permitted. Argo CD uses its own RBAC system, not the native RBAC offered by Kubernetes. When a user submits a request to the API server, the system examines the groups listed in the JWT’s “groups” claim and compares them against the RBAC policies defined within Argo CD. These policies consist of roles and rules, and if a match is found, the user is granted access to execute the request.
RBAC rules are customizable, allowing administrators to define granular permissions for different roles. This ensures that users or services only have access to the specific resources and operations required for their tasks. For example, roles can be limited to view-only permissions or restricted to a specific namespace or set of applications within a project.
TLS
Argo CD secures all network communications using Transport Layer Security (TLS). This includes not only external communications, such as interactions between users and the Argo CD API server, but also internal communications between the system’s components, such as the argocd-server
, argocd-repo-server
, and argocd-application-controller
. By default, all data exchanged between these services is encrypted.
Administrators have the option to enforce the use of TLS 1.2 for all communications by setting the --tlsminversion 1.2
flag on the API server. While TLS is enforced for most internal communications, Redis, which is used for caching and state storage, communicates over HTTP by default. However, TLS for Redis can be configured manually.
Securely Managing Git and Helm Repositories
Argo CD uses a dedicated service, known as the repo-server
, to manage Git and Helm repositories. The repo-server
is responsible for cloning and caching repositories, as well as generating Kubernetes manifests from them. Importantly, the repo-server
does not have any Kubernetes permissions or store sensitive credentials. By caching repositories locally, Argo CD optimizes performance.
In addition to raw YAML manifests, Argo CD supports the use of configuration management tools such as Helm and Kustomize. However, these tools introduce further risks if not properly managed. An attacker with write access to a repository could create malicious Helm charts or Kustomize files that attempt to read sensitive files stored out-of-tree, including files adjacent to the repository or on the repo-server itself.
For organizations that use sensitive information in these repositories, such as decryption keys or other secrets, it is crucial to ensure repository contents are always trusted. One way to mitigate this risk is to disable unused configuration management tools.
Sensitive Information Security
Argo CD takes measures to ensure sensitive data is protected. Sensitive information, such as cluster credentials, Git credentials, OAuth2 client secrets, and Kubernetes secret values, is never returned in API responses and is always redacted from logs.
For managing external clusters, Argo CD stores credentials as Kubernetes secrets in the argocd namespace. These secrets contain the API bearer tokens associated with the argocd-manager
service account created during the process of adding an external cluster to Argo CD. The tokens, along with other connection options such as TLS configuration and AWS role-arn information, are used to securely authenticate Argo CD to the external cluster.
In the case of Kubernetes versions 1.24 and later, which no longer create ServiceAccount tokens automatically, Argo CD creates non-expiring ServiceAccount tokens when adding a new cluster. Token rotation is handled manually, but Argo CD plans to introduce support for Kubernetes’ TokenRequest API in the future, which will allow for short-lived, automatically rotating tokens. To revoke access to an external cluster, administrators can delete the relevant RBAC artifacts and remove the cluster from Argo CD’s management.
Cluster RBAC
Argo CD typically operates using a clusteradmin role to manage and monitor the resources in a Kubernetes cluster. This role grants Argo CD wide-ranging privileges, enabling it to watch the entire cluster state and deploy resources as defined in Git repositories. However, granting Argo CD full cluster-wide privileges may not always be necessary or advisable from a security perspective.
Administrators can modify the ClusterRole used by Argo CD to restrict its write privileges to only specific namespaces or resource types that the organization wishes to manage with Argo CD. For example, Argo CD can be configured to have read-only access to certain cluster resources while retaining full deployment privileges in a limited set of namespaces. To further secure the system, specific types of resources can be excluded entirely from Argo CD’s management scope by adding them as excluded resources in the configuration.
Auditing
One of the advantages of using a GitOps-based tool like Argo CD is the built-in auditability of Git repositories. Every change made to application configurations, including what was changed, when, and by whom, is automatically recorded in the Git commit history. This provides a clear audit trail for configuration changes over time.
However, the Git history does not always correspond directly to actions taken in a Kubernetes cluster. For example, one user may make multiple changes to application manifests in Git, but another user may sync those changes to the cluster at a later time. To address this gap, Argo CD emits Kubernetes Events that track important application-related activities within the cluster. These events include information on who initiated specific actions, such as application creation, updates, or syncs.
For long-term retention of these events, organizations can integrate tools like Event Exporter or Event Router to store and analyze event data over extended periods.
Logging
Argo CD’s logging system is designed with security in mind. Logs are tagged with a “security” field to help identify events that may be relevant to security audits or incident investigations. These logs are categorized into five levels of severity, ranging from “Low” for normal access events to “Emergency” for unmistakably malicious activities, such as brute force attacks. Logs at higher severity levels typically indicate more urgent or dangerous issues that require immediate attention.
In addition to general API request logs, Argo CD refrains from logging sensitive information in certain scenarios, such as during session creation or cluster service operations. For security reasons, Argo CD does not log the IP addresses of clients directly, as these are typically handled by the proxy server in front of the Argo CD API server. Therefore, organizations should configure IP logging on the proxy level to capture this information.
Learn more in our detailed guide to Argo CD architecture
TIPS FROM THE EXPERT
In my experience, here are additional tips to further secure Argo CD setups:
- Use network policies for inter-component security: Employ Kubernetes Network Policies to restrict communication between Argo CD components, especially limiting ingress and egress to sensitive components like the repo-server. This helps mitigate lateral movement risks if one component is compromised.
- Automate security configuration checks: Use security tools, such as Open Policy Agent (OPA) or Kyverno, to enforce Argo CD configuration best practices, ensuring settings like RBAC, TLS, and namespace isolation follow strict security guidelines. Automating these checks ensures that security settings are consistently applied across environments.
- Configure IP allowlists on external access: Limit access to Argo CD’s API by creating IP allowlists to restrict access to known, trusted IP ranges. This reduces the risk of unauthorized access attempts from unknown locations.
- Implement secrets rotation and monitoring: Regularly rotate Argo CD’s secrets, such as automation tokens and JWTs, and monitor these credentials for unauthorized use. Set up alerts on suspicious activity, such as repeated failed token verifications, which could indicate attempted brute-force attacks.
- Isolate Helm/Kustomize operations with sandboxing: To prevent risks associated with Helm and Kustomize, containerize or sandbox these operations when Argo CD processes them. Tools like Pod Security Policies (PSP) or Open Policy Agent (OPA) can help restrict repo-server operations to mitigate the impact of malicious templates or scripts.
Overview of Past and Current Security Vulnerabilities in Argo CD
Argo CD has faced multiple security vulnerabilities, impacting various versions of the tool. Below is an overview of significant issues, along with fixes and workarounds where available:
- CVE-2020-1747 and CVE-2020-14343 – PyYAML library code execution vulnerability: The PyYAML library, used by Argo CD, was found to be susceptible to arbitrary code execution. This affected all versions of Argo CD prior to v1.5.8. However, Argo CD itself is not believed to be directly impacted as the vulnerabilities are limited to specific use cases involving AWS IAM authentication. The fix was included in version 1.5.8.
- CVE-2020-5260 – possible Git credential leak: This critical vulnerability, reported by Felix Wilhelm of Google Project Zero, exposed the possibility of Git credentials being leaked during operations. Though Argo CD does not use the vulnerable Git features by default, the project released patched versions (v1.4.3 and v1.5.2) to ensure protection for users who might configure Git credential helpers. Upgrading to these versions is strongly recommended.
- CVE-2020-11576 – user enumeration: In version v1.5.0, a medium-risk vulnerability allowed attackers to enumerate local usernames. This was mitigated in v1.5.1, and as a workaround, disabling local users and relying on SSO authentication was advised.
- CVE-2020-8828 – insecure default administrative password: A high-risk issue involved using the Argo CD pod name as the default admin password. This vulnerability allowed Kubernetes users to easily retrieve the password. Fixed in v1.8.0, the recommended mitigation is to change the admin password after setup or use SSO.
- CVE-2020-8827 – insufficient anti-brute force measures: Before v1.5.3, Argo CD lacked rate-limiting and anti-automation protections, exposing it to brute-force attacks on local admin accounts. Users are advised to upgrade to v1.5.3 or disable local accounts in favor of SSO for protection.
- CVE-2020-8826 – session fixation: This vulnerability, marked as high risk, related to the lack of expiry for authentication tokens used by built-in accounts. Mitigation strategies include regular password changes to invalidate these tokens.
- CVE-2018-21034 – sensitive information disclosure: Prior to v1.5.0, authenticated users could exploit Argo CD APIs to retrieve sensitive secrets stored in Git repositories. This medium-risk vulnerability was addressed in v1.5.0, with no available workarounds for earlier versions.
Best Practices for Securing Argo CD
Use a Dedicated Project for the Control Plane
Using a dedicated project for managing the Argo CD control plane is crucial for maintaining the integrity of your infrastructure. Without this separation, sharing the control plane project with other applications can create vulnerabilities, as applications would have access to Argo CD’s namespace and could unintentionally or maliciously modify workloads.
To properly configure the dedicated project, ensure that the destination fields of the control plane application resources are set to the local cluster (https://kubernetes.default.svc) and to the namespace where Argo CD is deployed. The source repositories specified in this project should be tightly controlled and managed only by Argo CD administrators.
By isolating the control plane in its own project, you restrict the ability of other projects to affect its operations. No other project should be able to target the same namespace or cluster that Argo CD uses. Administrators must also secure the Git repositories associated with this project by enforcing best practices such as private repositories, branch protection, signed commits, and requiring code reviews for sensitive changes like those affecting access control or role bindings.
Argo Resources are for Argo Admins Only
A critical best practice is ensuring that only Argo CD administrators can manage resources within the argoproj.io API group, such as Application and ApplicationSet. These resources are fundamental to how Argo CD operates, and allowing non-admin users to manage them increases the risk of security incidents.
Specifically, Application resources, which define how applications are deployed and synced to clusters, should only be deployed in the namespace where the Argo CD instance is running, and their management should be restricted to trusted administrators.
Allowing non-admin users to access these resources can lead to serious security implications, as they could inadvertently alter the configuration or introduce vulnerabilities. For example, modifying the destination namespace or cluster settings of an Application resource could disrupt the intended deployment process or expose sensitive systems to unauthorized access.
Block ClusterRoleBindings in Most Projects
ClusterRoleBindings, which grant permissions across an entire Kubernetes cluster, should be carefully controlled. Allowing applications to create or modify ClusterRoleBindings can lead to privilege escalation, where a service account might gain broad permissions and execute actions outside its intended scope. To avoid this, it’s advisable to block ClusterRoleBindings in most Argo CD projects unless explicitly required by cluster administrators.
This measure forces meaningful discussions between product teams and cluster administrators before any elevated permissions are granted. These discussions often lead to safer, more granular permission models, such as using dedicated service accounts with minimal roles.
By limiting the ability to create ClusterRoleBindings, you reduce the potential for an application to accidentally or intentionally compromise the cluster. Exceptions to this rule should be made cautiously and only in scenarios where a project managed by cluster administrators requires ClusterRoleBindings for legitimate administrative tasks.
Narrow Roles on Remote Clusters
When Argo CD is used to manage multiple clusters, limiting its permissions on remote clusters is vital for minimizing security risks. A common mistake is granting the Argo CD control plane full cluster-wide permissions across all managed clusters, which can have catastrophic consequences if the control plane is compromised. Instead, each remote cluster should have a dedicated service account with limited roles that only apply to the specific namespaces Argo CD needs to manage.
To configure this setup, work closely with cluster administrators to define the exact namespaces that Argo CD should have access to. This ensures that even if the control plane is compromised, the damage is contained within a limited scope.
For example, granting Argo CD full permissions to deploy applications in a specific namespace might be acceptable, but avoid giving it access to sensitive resources like secrets unless absolutely necessary. Service accounts should be used to generate kubeconfig
files for each cluster, and these credentials should be rotated regularly.
Use RBAC in Multi Tenant/Multi-User Environments
In multi-tenant or multi-user environments, Argo CD’s Role-Based Access Control (RBAC) system plays a crucial role in ensuring secure, controlled access to resources. RBAC allows administrators to define granular permissions for different users or groups, ensuring they only have access to the resources necessary for their tasks.
Argo CD provides two built-in roles: role:readonly
and role:admin
, with read-only and unrestricted access, respectively. However, these default roles are often insufficient for complex environments where users require different levels of access. Administrators can create custom roles with more specific permissions by defining policies in the RBAC configuration.
For multi-tenant environments, it’s important to restrict access to sensitive resources, such as applications, repositories, and clusters, by assigning policies that limit which users or groups can manage them. Policies are structured using Casbin syntax, allowing administrators to define what actions (e.g., create
, delete
, update
) a role can perform on a specific resource type.
In addition, isolating application projects by defining project-specific RBAC roles ensures that users only have access to applications within their designated projects. This prevents users from one team or tenant from accidentally or maliciously affecting resources owned by another. For example, an administrator can configure policies to allow a user to view and modify applications only in their assigned project, while restricting access to other projects and namespaces.
Have a CVE Response Plan Ready
As described above, Argo CD is not immune to vulnerabilities, and it’s critical to have a well-defined plan for responding to Common Vulnerabilities and Exposures (CVEs). A proactive CVE response plan ensures that you can quickly assess the impact of a vulnerability and implement necessary mitigations to protect your infrastructure.
Argo CD publishes its CVEs as GitHub security advisories, making it easy for administrators to stay informed about the latest vulnerabilities. Monitoring these advisories and reviewing security issues labeled as “security” helps you stay ahead of potential risks.
When a CVE is announced, the first step is to apply the recommended patches or mitigations as quickly as possible. This may involve updating Argo CD to a more secure version, disabling affected features, or tightening configuration settings. However, patching alone is not sufficient; administrators must also investigate whether the vulnerability was exploited. This involves reviewing audit events and logs, particularly those related to applications managing the control plane, to detect any unusual or unauthorized activity.
Additionally, regularly reviewing roles, role bindings, and other sensitive configurations in the cluster hosting the control plane ensures that no unauthorized access was gained during the vulnerability window.
Codefresh: A Modern, Argo-Based CI/CD Platform
The Codefresh platform, powered by Argo, combines the best of the open-source with an enterprise-grade runtime allowing you to fully tap the power of Argo Workflows, Events, CD, and Rollouts. It provides teams with a unified GitOps experience to build, test, deploy, and scale their applications.
You can use Codefresh for your platform engineering initiative either as a developer portal or as the machinery that takes care of everything that happens in the developer portal. Your choice depends on how far your organization has adopted Kubernetes and micro-services
Codefresh is a next generation CI/CD platform designed for cloud-native applications, offering dynamic builds, progressive delivery, and much more.