Kubernetes (K8S) comes with a set of objects out of the box to manage containers, but what if you need to manage something outside of your regular Kubernetes scope? That’s where custom resource definitions (CRDs) come into the page.
CRD is a powerful feature of Kubernetes and allows you to extend native APIs, allowing you to create your own resource types. Whether it’s a new application or a unique service not designed to handle by default, think of it like a custom tool you can add to Kubernetes to manage what you need.
In this article, we dive into what CRDs are, where you can use them, and how to get started with creating your own. We also explain how to manage them and share some best practices for using CRD effectively.
What is a Kubernetes Custom Resource Definition (CRD)?
Kubernetes Custom Resource Definitions (CRDs) allow you to define your own custom objects that work in the same way as default Kubernetes resources (such as pods and deployments). Define these custom resources to enable Kubernetes to manage a wider range of use cases beyond built-in objects.
For example, you can define a CRD to manage database instances that can automate the provisioning and scaling of databases within Kubernetes itself. CRD is especially useful for building Kubernetes operators, the controller that automates the lifecycle of complex applications.
The difference between Kubernetes API and CRD
In Kubernetes, the API (Application Programming Interface) defines programmatic interactions between system components. Provides predefined endpoints for managing resources such as PODs, services, deployments, and more.
Custom Resource Definition (CRD) is an extension to the Kubernetes API that allows you to define custom resource types beyond built-in resource types. CRD allows developers to introduce application-specific abstractions while still maintaining compatibility with Kubernetes’ declarative management model.
The Kubernetes API extension provides deeper integration by modifying or extending the Kubernetes API server itself. This includes mechanisms such as aggregated API servers and admission webhooks. This allows you to apply custom logic at the API level, such as validation, authentication, and miso control.
While CRD is easier to manage and implement, API extensions offer more flexibility and control, making them suitable for advanced use cases where custom logic or specialized rules are required.
Differences between ConfigMaps and CRD in Kubernetes
ConfigMaps holds insensitive configuration data to key value pairs that allow applications to separate code from configuration. This allows the pod to access external settings without having to modify the container image.
Essentially, ConfigMaps is used to store external configuration data for applications, and CRD extends the Kubernetes API to support custom resources. This difference makes CRDs good for creating reusable and extensible Kubernetes objects, but they are suitable for storing environment-specific settings.
Also Read: Data Management in Kubernetes using Portworx
What are the common use cases for Kubernetes CRD?
Common use cases include:
Build custom operators and controllers
CRD helps you create operators that automate complex tasks such as installation, upgrade, and scaling. They manage the application lifecycle without manual effort.
Manage application workloads
CRDS defines custom resources according to specific applications such as Kafkacluster and Elasticsearch Cluster. This simplifies deployment, configuration and scaling.
Automate infrastructure management
Tools like Crossplane use CRD to represent cloud infrastructures such as databases, storage, and networking. This allows teams to declaratively manage their infrastructure within Kubernetes.
Enforce policies and compliance
CRD works with the controller to enforce security policies, compliance standards, and resource constraints. These rules apply automatically across the cluster.
Supports CI/CD pipelines
The Kubernetes-Native CI/CD tool defines CRDs to manage deployment workflows. This ensures an automated, reproducible, consistent release process.
Also Read: Optimizing CI/CD Pipelines using DevOps Best Practices
How can I view a list of all existing CRDs in Kubernetes?
Especially if you are using a specific tool, it is possible that your cluster already has CRDS installed. To view them, just run it.
kubectl get crdIf you want specific CRD details, you can use:
kubectl describe crd Some Kubernetes tools will be installed automatically when you set up CRD. This allows the tool to provide functionality beyond what Kubernetes offers by default. Here are some examples:
- Helm Chart: Many helm charts come with CRDs to enable additional features (e.g. Cert-Manager, Istio, Argocd).
- operator:Kubernetes operators typically use CRD to manage custom resources (such as Prometheus operators, FluxCDs).
- kustomize: A tool that can be used to install CRD along with other Kubernetes resources.
For example, when you install Prometheus on a cluster, the corresponding Prometheus CRD will be automatically installed and you can use Prometheus-specific resources.
Also Read: Top DevOps Tools for Seamless Salesforce CI/CD Integration
How to create a Kubernetes CRD?
To extend Kubernetes with custom resources, you can define a custom resource definition (CRD). This allows Kubernetes to recognize and manage new resource types beyond built-in resources, such as pods and deployments. Once you have defined and applied CRDs, you can create instances of custom resources (CRs) and manage them.
Here is an example of creating a CRD in Kubernetes to manage switches in a data center: Most use cases can take advantage of existing CRDs available online, so they should be created from scratch if necessary.
Define the CRD YAML file
The first step is to define the YAML file for the CRD. This file specifies the API groups, versions, and schemas for the custom resource.
example: switchcrd.yaml (CRD definition for the switch)
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: switches.datacenter.com # Name must match the format: .
spec:
group: datacenter.com # Group name for the REST API (e.g., /apis//)
versions:
- name: v1
served: true # Whether this version is available
storage: true # The storage version, only one version can be marked as storage
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
dataCenter:
type: string
rack:
type: integer
type:
type: string
scope: Cluster # Can be either Namespaced or Cluster
names:
plural: switches # Plural name used in the URL (/apis///)
singular: switch # Singular name used in the CLI and for display
kind: Switch # The kind field, used in the resource manifests
shortNames:
- sw # Short name for use in the CLI
Apply CRD to Kubernetes
Once CRD YAML is ready, apply it to your Kubernetes cluster using the following command:
kubectl apply -f switchcrd.yaml
This will register a new CRD in Kubernetes and allow you to create instances of your custom resource.
Check the CRD
To verify that the CRD was successfully applied, list the CRDs in the cluster.
kubectl get crd
Now that you have the CRD in place, you can check if your custom resources are available.
kubectl get switches
At this point, you will see an empty list because you defined the CRD, but no custom resource (switch) has been created yet.
Create a custom resource (CR)
Next, create a custom resource based on the CRD. In this case, let’s define the switch to be called Switch 1.
example: switch1.yaml (Custom resource definition for the switch)
apiVersion: datacenter.com/v1
kind: Switch
metadata:
name: switch1
spec:
dataCenter: uksouth
rack: 280
type: physical
Apply custom resources to Kubernetes.
kubectl apply -f switch1.yaml
Check your custom resources
Now that you have created a custom resource, you can list all the switches.
kubectl get switches
At this point, the switch is now part of the Kubernetes API. However, this resource does not perform any actions on its own. This is a record with the specifications defined.
Automate custom resources with controllers
To make a custom resource work, Custom Controller. A custom controller is an application or script that listens to changes in a custom resource and performs actions accordingly. For example, you can use tools such as Kubebuilder, Operator SDKor custom GO code to create these controllers.
The custom controller monitors the Kubernetes API for changes to the switch object and takes appropriate actions based on the logic defined. This introduces automation to manage the state of custom resources and integrate with Kubernetes clusters in a seamless way.
By following these steps, you can create a CRD, deploy custom resources, and automate its management through a custom controller. This approach provides a flexible and scalable way to manage custom Kubernetes resources, allowing you to extend Kubernetes to your own requirements.
Also Read: How to manage and secure virtualized and containerized environments with scale computing and Lubric?
How can I manage Kubernetes CRDS?
Effective management of custom resource definitions (CRDs) in Kubernetes is important to maintain resource stability and integrity. This includes being careful when updating CRDs, handling the version properly and ensuring that deletion is done safely. Without proper management, you can face problems such as schema conflicts, orphan resources, or unexpected failures in applications.
Update the CRD
When updating a CRD, it is important to avoid destroying existing resources. Updates to CRD are usually safe if the changes are not disruptive, such as adding new options fields. However, Kubernetes performs schema validation, so even small changes such as constraints and changes to default values ​​can cause problems if not properly tested.
Can be used kubectl apply or kubectl replace The command to apply the update. It’s best to preview it with before making any changes kubectl diff Or use kubectl apply --dry-run=client To ensure backward compatibility. We also recommend that you use Openapi V3 to validate your schema.
example:
Here is an example of updating a CRD:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: widgets.example.com
spec:
group: example.com
names:
kind: Widget
listKind: WidgetList
plural: widgets
singular: widget
scope: Namespaced
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
size:
type: string
To apply this update, use the following:
kubectl apply -f widget-crd.yaml
If you need to make changes to the structural schema (for example, deleting fields), Kubernetes will not allow these changes in the current version x-kubernetes-preserve-unknown-fields It was enabled in advance.
Version CRDS
The CRDS version is important to enable incremental improvements while ensuring API stability. Kubernetes supports versioning by defining multiple versions in versions Field.
For example, if you want to add a new version v2 Maintaining v1 Version, you define versions Fields like this:
versions:
- name: v1
served: true
storage: false
- name: v2
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
size:
type: string
color: # New field in v2
type: string
To apply the version update, run it.
kubectl apply -f widget-crd.yaml
It is important to check which version you are using before blaming it.
kubectl get crd widgets.example.com -o yaml | grep versions -A 5
Set when you criticize the version storage: false If it’s that version, it will be kept served: true It allows for a temporary smooth transition. Once all resources have migrated to the new version, you can safely remove the deprecated version to reduce overhead.
Delete crds
Deleting a CRD can cause data loss as all associated resources are deleted. Therefore, it is essential that your critical workloads do not depend on CRD before deleting them.
Kubernetes does not automatically back up CRDs, so you must export your custom resource data before you delete it.
kubectl get myresources.example.com
If resources still exist, consider backing them up first. You can also use the finalizer to prevent CRDs from being deleted before the required cleanup is done.
Example of adding a final riser to prevent deletion:
apiVersion: example.com/v1
kind: MyResource
metadata:
name: example-resource
finalizers:
- prevent-deletion
To safely remove a CRD, use:
kubectl delete crd myresources.example.com
If the deletion hangs, check the reason for the block.
kubectl describe crd myresources.example.com
kubectl get events --field-selector involvedObject.name=myresources.example.com
kubectl logs
If you encounter problems, you can force the finalizer to be removed.
kubectl patch myresource example-resource -p '{"metadata":{"finalizers":[]}}' --type=merge
Follow these steps to manage updates, version versions, and deletions to ensure that CRD is handled correctly without destroying resources or causing unexpected issues in your Kubernetes environment.
Best Practices when Using Kubernetes CRD
There are some best practices to help you manage your CRD effectively.
- Implement the controller: Use the controller to automate the management and coordination of custom resources.
- Secure access: Define appropriate RBAC policies to restrict access to sensitive CRDs.
- Track resource status: Uses status subresources to track the status of the resource separately from the specs.
- Designed for stabilityUse versioning to safely introduce changes to resources.
- Validate with Openapi schema: Use OpenAPI schema validation to ensure that the resource is configured correctly.
- Follow Kubernetes rules: Stick to standard naming conventions and resource structures for consistency.
I’ll summarize
Custom Resource Definitions (CRDs) are Kubernetes game changers that allow you to define and manage custom resources that meet your needs. Whether building automation, defining custom workloads, or managing infrastructure, CRD provides the flexibility to extend Kubernetes beyond built-in resources.
Follow best practices and use the right tools to leverage CRDS to streamline Kubernetes management and unlock new possibilities for your applications.
