lineapy.db package

Submodules

lineapy.db.db module

class lineapy.db.db.RelationalLineaDB(url: str)

• Note that LineaDB coordinates with asset manager and relational db.

  • The asset manager deals with binaries (e.g., cached values) The relational db deals with more structured data, such as the Nodes and edges.

• Also, at some point we might have a “cache” such that the readers don’t have to go to the database if it’s already loaded, but that’s low priority.

artifact_in_db(node_id: LineaID, execution_id: LineaID, name: str, version: int) → bool

Returns true if the artifact is already in the DB.

close()

Close the database connection.

commit() → None

End the transaction and commit the changes.

get_all_artifacts() → List[ArtifactORM]

Used by the catalog to get all the artifacts

get_artifact_by_name(artifact_name: str, version: Optional[int] = None) → ArtifactORM

Gets the most recent artifact with a certain name. If a version is not specified, it will return the most recent version sorted by date_created

get_artifacts_for_session(session_id: LineaID) → List[ArtifactORM]

Gets a code slice for an artifact by name, assuming there is only one artifact with that name,

get_latest_artifact_version(artifact_name: str) → int

Get the latest version number of an artifact. If the artifact does not exist, it will return -1

get_libraries_for_session(session_id: LineaID) → List[ImportNodeORM]

Gets all dependencies for a session, assuming all the libs in a particular session will be required to set up a new env.

TODO: I think this distinct is still broken, because we want to make it distinct on a subset of columns: session_id, name, and version.

get_node_by_id(linea_id: LineaID) → Union[ImportNode, CallNode, LiteralNode, LookupNode, MutateNode, GlobalNode]

Returns the node by looking up the database by ID SQLAlchemy is able to translate between the two types on demand

get_node_value_path(node_id: LineaID, execution_id: LineaID) → Optional[str]

Get the path to the value of the artifact. :param other: Additional argument to let you query another artifact’s value path.

This is set to be optional and if its not set, we will use the current artifact

get_nodes_for_session(session_id: LineaID) → List[Union[ImportNode, CallNode, LiteralNode, LookupNode, MutateNode, GlobalNode]]

Get all the nodes associated with the session, which does

NOT include things like SessionContext

node_value_in_db(node_id: LineaID, execution_id: LineaID) → bool

Returns true if the node value is already in the DB.

write_source_code(source_code: SourceCode) → None

Writes a source code object to the database.

It first has to convert it to a SourceCodeORM object, which has the fields inlined instead of a union

lineapy.db.relational module

This file contains the ORM versions of the graph node in types.py.

Pydantic allows us to extract out a Dataclass like object from the ORM, but not let us directly write to the ORM.

Relationships

Warning

non exhaustive list

SessionContext - ImportNode (One to Many) - HardwareSpec (Many to One)

Node - SessionContext (Many to One)

CallNode - Node (Many to Many)

class lineapy.db.relational.ArtifactORM(**kwargs)

An artifact is a named pointer to a node.

class lineapy.db.relational.BaseNodeORM(**kwargs)

node.source_code has a path value if node.session.environment_type == “script” otherwise the environment type is “jupyter” and it has a jupyter execution count and session id, which is equal to the node.session

Note

• Because other nodes are inheriting from BaseNodeORM, finding a node based on its id is easy (something like the following):

  session.query(BaseNodeORM).filter(BaseNodeORM.id == linea_id)
  

• Each node inheriting from BaseNodeORM must have non null values for all of lineno, col_offset, end_lineno, end_col_offset and source_code_id or nulls for all of them.

class lineapy.db.relational.CallNodeORM(**kwargs)

class lineapy.db.relational.ExecutionORM(**kwargs)

An execution represents one Python interpreter invocation of some number of nodes

class lineapy.db.relational.GlobalNodeORM(**kwargs)

class lineapy.db.relational.GlobalReferenceORM(**kwargs)

class lineapy.db.relational.ImplicitDependencyORM(**kwargs)

class lineapy.db.relational.ImportNodeORM(**kwargs)

class lineapy.db.relational.KeywordArgORM(**kwargs)

class lineapy.db.relational.LiteralNodeORM(**kwargs)

class lineapy.db.relational.LookupNodeORM(**kwargs)

class lineapy.db.relational.MutateNodeORM(**kwargs)

class lineapy.db.relational.NodeValueORM(**kwargs)

A node value represents the value of a node during some execution.

It is uniquely identified by the node_id and execution_id.

The following invariant holds: value.node.session == value.execution.session

class lineapy.db.relational.PositionalArgORM(**kwargs)

class lineapy.db.relational.SessionContextORM(**kwargs)

class lineapy.db.relational.SourceCodeORM(**kwargs)

lineapy.db.utils module

class lineapy.db.utils.FilePickler

Tries to pickle an object, and if it fails returns None.

Module contents