lineapy.api package

Submodules

lineapy.api.api module

User facing APIs.

lineapy.api.api.catalog() → LineaCatalog[source]
Returns

An object of the class LineaCatalog that allows for printing and exporting artifacts metadata.

Return type

LineaCatalog

lineapy.api.api.get(artifact_name: str, version: Optional[int] = None) → LineaArtifact[source]

Gets an artifact from the DB.

Parameters

  • artifact_name (str) – name of the artifact. Note that if you do not remember the artifact, you can use the catalog to browse the options

  • version (Optional[str]) – version of the artifact. If None, the latest version will be returned.

Returns

returned value offers methods to access information we have stored about the artifact

Return type

LineaArtifact

lineapy.api.api.save(reference: object, name: str) → LineaArtifact[source]

Publishes the object to the Linea DB.

Parameters

  • reference (Union[object, ExternalState]) – The reference could be a variable name, in which case Linea will save the value of the variable, with out default serialization mechanism. Alternatively, it could be a “side effect” reference, which currently includes either lineapy.file_system or lineapy.db. Linea will save the associated process that creates the final side effects. We are in the process of adding more side effect references, including assert statements.

  • name (str) – The name is used for later retrieving the artifact and creating new versions if an artifact of the name has been created before.

Returns

returned value offers methods to access information we have stored about the artifact (value, version), and other automation capabilities, such as to_pipeline().

Return type

LineaArtifact

lineapy.api.api.to_pipeline(artifacts: List[str], framework: str = 'SCRIPT', pipeline_name: Optional[str] = None, dependencies: Dict[str, Set[str]] = {}, pipeline_dag_config: Optional[AirflowDagConfig] = {}, output_dir: Optional[str] = None) → Path[source]

Writes the pipeline job to a path on disk.

Parameters

  • artifacts – list of artifact names to be included in the DAG.

  • framework – ‘AIRFLOW’ or ‘SCRIPT’

  • pipeline_name – name of the pipeline

  • dependencies – tasks dependencies in graphlib format {‘B’:{‘A’,’C’}}, this means task A and C are prerequisites for task B.

  • output_dir_path – Directory of the DAG and the python file it is saved in; only use for PipelineType.AIRFLOW

Returns

string containing the path of the DAG file that was exported.

Module contents