5.3 Compose a pipeline

A pipeline is a machine learning workflow, consisting of one or more steps, to deploy code using Docker containers. By specifying the output of one step as the input of another step, the user can create a full pipeline formed with a computed acyclic graph (DAG).

ℹ️ Info: For the moment, the pipelines are only single-step. We are actively working on the integration of multi-steps for the next versions of the platform.

To deploy ML code in production, you need to go through a pipeline. So you have to go through a single-step pipeline in any case to perform an endpoint or another type of deployment.

Summary:

1. Compose a mono-step pipeline

2. Delete a pipeline

3. Find and get pipeline information

Function name	Method	Return type	Description
create_pipeline	create_pipeline(pipeline_name, step_name)	dict[str, str]	Create a pipeline containing a single step.
get_pipeline	get_pipeline(pipeline_name)	dict	Get a single pipeline if it exists.
delete_pipeline	delete_pipeline(pipeline_name, force_deployments_deletion=False)	dict	Delete a pipeline identified by its name and ID.
list_pipelines	list_pipelines()	list of dict	Get the list of all pipelines.

5.3.1 Create a mono-step pipeline

5.3.1.1 SDK function pipeline

Function definition

pipeline = sdk.create_pipeline(
   pipeline_name="my_pipeline",
   step_name="my_step",
)

Parameters

• pipeline_name (str) – Name of the pipeline to create.

• step_name (str) – Name of the step to include in the pipeline.

  Note

  The step should have the status “Ready” before being used to create the pipeline.

Returns

Created pipeline represented as dict (with key “id”) as dict[str, str] object.

• “pipeline_name” (str): Pipeline name.

• “created_at” (str): Pipeline date of creation.

• “steps” (list): List of step names.

• “open_inputs” (list): List of all input of step.

• “open_output” (list): List of all output of step.

5.3.1.2 Information about pipeline store

Pipeline can have multiple inputs and outputs, or no one. In fact, it’s dependent on the inputs and outputs of step inside. All input and output will be the same as the step inside. So, you have nothing to configure on pipeline creation for input and output.

5.3.2 Get information pipeline

5.3.2.1 Get information about one pipeline

Function definition

Get all information about one pipeline, referred by its name.

CraftAiSdk.get_pipeline(pipeline_name)

Parameters

• pipeline_name (str) – Name of the pipeline to get.

Returns

The pipeline information in a dict or None if the pipeline does not exist.

• “id” (str): Pipeline id.

• “name” (str): Pipeline name.

• “created_at” (str): Pipeline date of creation.

• “steps” (list): List of step names.

5.3.2.2 Get all pipelines

Function definition

Get the list of all pipelines on your environment.

CraftAiSdk.list_pipelines()

Returns

List of pipelines represented as dict with keys :

• “pipeline_name” (str): Name of pipeline

• “created_at” (str): Create date of the pipeline.

5.3.3 Delete a pipeline

5.3.3.1 Function definition

Delete a pipeline from the current environment.

CraftAiSdk.delete_pipeline(pipeline_name, force_deployments_deletion=False)

Parameters

• pipeline_name (str) – Name of the pipeline.

• force_deployments_deletion (bool, optional) – if True, the associated endpoints will be deleted too. Defaults to False.

Returns

The deleted pipeline and associated deleted endpoints in a dict. The returned dict contains two keys:

• “pipeline”(dict): Deleted pipeline represented as dict (with keys “id” and “name”).

• “endpoints” (list): List of deleted endpoints represented as dict (with keys “id” and “name”).

Warning

You can’t delete a pipeline that is used in a Deployment

5.3.4 Run a pipeline

Now that your pipeline is created you can run it directly from the SDK. Or you can configure a deployment.