ll.vsql – Half an ORM
Normally an object-relational mapper consist of:
An infrastructure for specifying which records to fetch from the database. This can be as simple as specifying a primary key value, or as complex as a complete query language.
A way of mapping those records to objects and fields to attributes.
vSQL provides the first part only. vSQL does this by using a subset of UL4 expressions retargeted for generating SQL expressions used in SQL queries. So the end result of using vSQL always is a SQL query that you can execute normally.
Currently only Oracle is supported.
The purpose of vSQL is not to abstract away differences between the relational database and the object oriented world or to shield the developer from having to deal with SQL or from handling the result of database queries.
Instead vSQL provides a safe query language that can be used to provided a publicly accessible interface to database content. This eliminates the risky parts of query construction, effectively preventing SQL injection attacks, while offering the expressive power of an ORM without the overhead.
Example
To create an SQL query via vSQL expressions in the simplest form we’re using the
class Query like this:
from ll import vsql
q = vsql.Query()
# Define the expression we want
q.select_vsql("('foo'.upper() + 'bar'.lower())[:3]")
q.select_vsql("now() + years(3)")
# Output the SQL query
print(q.sqlsource())
this outputs:
select
vsqlimpl_pkg.slice_str((upper('foo') || lower('bar')), null, 3) /* ('foo'.upper() + 'bar'.lower())[:3] */,
vsqlimpl_pkg.add_datetime_months(sysdate, (12 * 3)) /* now() + years(3) */
from
dual
Selecting from tables
If you want to select records from a table, the Query object needs to
know about the structure of the table, and needs a variable that can be used
to refer to records of that table. If the table looks like this:
create table person
(
per_id integer,
per_firstname varchar2(100),
per_lastname varchar2(100),
per_birthday date
);
the table definition for vSQL can be defined like this:
person_table = vsql.Group("person")
person_table.add_field("id", vsql.DataType.INT, "{a}.per_id")
person_table.add_field("firstname", vsql.DataType.STR, "{a}.per_firstname")
person_table.add_field("lastname", vsql.DataType.STR, "{a}.per_lastname")
person_table.add_field("birthday", vsql.DataType.DATE, "{a}.per_birthday")
Selecting from this table then works like this:
from ll import vsql
person_table = vsql.Group("person")
person_table.add_field("id", vsql.DataType.INT, "{a}.per_id")
person_table.add_field("firstname", vsql.DataType.STR, "{a}.per_firstname")
person_table.add_field("lastname", vsql.DataType.STR, "{a}.per_lastname")
person_table.add_field("birthday", vsql.DataType.DATE, "{a}.per_birthday")
q = vsql.Query(
p=vsql.Field("p", vsql.DataType.INT, refgroup=person_table)
)
# We want to force the query to select from `person`,
# even if we don't select any fields
q.register_vsql("p")
# Specify which fields we want returned
q.select_vsql("p.firstname + ' ' + p.lastname")
q.select_vsql("p.birthday")
# Return anly people born 1990 or later
q.where_vsql("p.birthday > @(1990-01-01)")
# Youngest first
q.orderby_vsql("p.birthday", "desc")
# Output the SQL query
print(q.sqlsource())
this outputs:
select
((t1.per_firstname /* p.firstname */ || ' ') || t1.per_lastname /* p.lastname */) /* p.firstname + ' ' + p.lastname */,
t1.per_birthday /* p.birthday */
from
person t1 /* p */
where
decode(vsqlimpl_pkg.cmp_datetime_datetime(t1.per_birthday /* p.birthday */, to_date('1990-01-01', 'YYYY-MM-DD')), 1, 1, null, null, 0) = 1 /* p.birthday > @(1990-01-01) */
order by
t1.per_birthday /* p.birthday */ desc
Using foreign key references
When you have foreign key fields that reference other tables,
you can use them too. When we assume that our table person
looks like this:
create table person
(
per_id integer,
per_firstname varchar2(100),
per_lastname varchar2(100),
per_birthday date,
com_id integer
);
with the new field com_id referencing the table company
that looks like this:
create table company
(
com_id integer,
com_name varchar2(100)
);
then we can define the table structure and query connected fields, as the following code demonstrates:
from ll import vsql
company_table = vsql.Group("company")
company_table.add_field("id", vsql.DataType.INT, "{a}.com_id")
company_table.add_field("name", vsql.DataType.STR, "{a}.com_name")
person_table = vsql.Group("person")
person_table.add_field("id", vsql.DataType.INT, "{a}.per_id")
person_table.add_field("firstname", vsql.DataType.STR, "{a}.per_firstname")
person_table.add_field("lastname", vsql.DataType.STR, "{a}.per_lastname")
person_table.add_field("birthday", vsql.DataType.DATE, "{a}.per_birthday")
person_table.add_field("company", vsql.DataType.INT, "{a}.com_id", "{m}.com_id = {d}.com_id", company_table)
q = vsql.Query(
p=vsql.Field("p", vsql.DataType.INT, refgroup=person_table)
)
# We want to force the query to select from `person`,
# even if we don't select any fields
q.register_vsql("p")
# Specify which fields we want returned
q.select_vsql("p.firstname + ' ' + p.lastname")
q.select_vsql("p.birthday")
q.select_vsql("p.company.name")
# Return anly people born 1990 or later
q.where_vsql("p.birthday > @(1990-01-01)")
# Youngest first
q.orderby_vsql("p.birthday", "desc")
# Output the SQL query
print(q.sqlsource())
This outputs:
select
((t1.per_firstname /* p.firstname */ || ' ') || t1.per_lastname /* p.lastname */) /* p.firstname + ' ' + p.lastname */,
t1.per_birthday /* p.birthday */,
t2.com_name /* p.company.name */
from
person t1 /* p */,
company t2 /* p.company */
where
t1.com_id = t2.com_id /* p.company */ and
decode(vsqlimpl_pkg.cmp_datetime_datetime(t1.per_birthday /* p.birthday */, to_date('1990-01-01', 'YYYY-MM-DD')), 1, 1, null, null, 0) = 1 /* p.birthday > @(1990-01-01) */
order by
t1.per_birthday /* p.birthday */ desc
vSQL standard library
Many vSQL operations can not be converted to simple SQL expressions.
For these vSQL uses an Oracle package vsqlimpl_pkg that contains
the “vSQL standard library”. This packages is available from Github at
https://github.com/LivingLogic/LivingLogic.Oracle.ul4
Module content
- class ll.vsql.sqlliteral[source]
Bases:
strInternal marker class that can be used to specify that its value should be treated as literal SQL.
- ll.vsql.comment(s: str) str[source]
Return an SQL comment with the content
s.I.e.
comment("foo")returns"/* foo */".
- class ll.vsql.Repr[source]
Bases:
objectBase class that provides functionality for implementing
__repr__()and_repr_pretty_()(used by IPython).
- class ll.vsql.DataType[source]
Bases:
EnumThe datatypes supported in vSQL expressions.
Possible values are:
NULLBOOLINTNUMBERSTRCLOBCOLORGEODATEDATETIMEDATEDELTADATETIMEDELTAMONTHDELTANULLLISTINTLISTNUMBERLISTSTRLISTCLOBLISTDATELISTDATETIMELISTNULLSETINTSETNUMBERSETSTRSETDATESETDATETIMESET
- classmethod compatible_to(given: DataType, required: DataType) Error | None[source]
Check whether the type
givenis compatible torequired.If
requiredisNoneeverygiventype is accepted. Otherwise the types must be compatible (for exampleDataType.INTis compatible toDataType.NUMBER, but not the other way around). Every type is compatible to itself.If
givenis not compatible torequiredthe appropriate error value is returned, otherwiseNoneis returned.
- class ll.vsql.NodeType[source]
Bases:
EnumThe different types of vSQL abstract syntax tree nodes.
This corresponds to the different subclasses of
AST.Possible values are:
FIELDCONST_NONECONST_BOOLCONST_INTCONST_NUMBERCONST_STRCONST_CLOBCONST_DATECONST_DATETIMECONST_TIMESTAMPCONST_COLORLISTSETCMP_EQCMP_NECMP_LTCMP_LECMP_GTCMP_GEBINOP_ADDBINOP_MULBINOP_SUBBINOP_FLOORDIVBINOP_TRUEDIVBINOP_MODBINOP_ANDBINOP_ORBINOP_CONTAINSBINOP_NOTCONTAINSBINOP_ISBINOP_ISNOTBINOP_ITEMBINOP_SHIFTLEFTBINOP_SHIFTRIGHTBINOP_BITANDBINOP_BITORBINOP_BITXORTERNOP_SLICEUNOP_NOTUNOP_NEGUNOP_BITNOTTERNOP_IFATTRFUNCMETH
- exception ll.vsql.VSQLError[source]
Bases:
ExceptionBase class of exceptions that can happend when compiling vSQL expressions.
- class ll.vsql.Error[source]
-
The types of errors that can lead to invalid vSQL AST nodes.
Note that some of those can not be produced by the Python implementation.
Possible values are:
SUBNODEERRORNODETYPEARITYSUBNODETYPESFIELDCONST_BOOLCONST_INTCONST_NUMBERCONST_DATECONST_DATETIMECONST_TIMESTAMPCONST_COLORNAMELISTUNSUPPORTEDTYPESSETMIXEDTYPESSETUNSUPPORTEDTYPESDATATYPE_NULLDATATYPE_BOOLDATATYPE_INTDATATYPE_NUMBERDATATYPE_STRDATATYPE_CLOBDATATYPE_COLORDATATYPE_DATEDATATYPE_DATETIMEDATATYPE_DATEDELTADATATYPE_DATETIMEDELTADATATYPE_MONTHDELTADATATYPE_NULLLISTDATATYPE_INTLISTDATATYPE_NUMBERLISTDATATYPE_STRLISTDATATYPE_CLOBLISTDATATYPE_DATELISTDATATYPE_DATETIMELISTDATATYPE_NULLSETDATATYPE_INTSETDATATYPE_NUMBERSETDATATYPE_STRSETDATATYPE_DATESETDATATYPE_DATETIMESET
- class ll.vsql.Field[source]
Bases:
ReprA
Fieldobject describes a database field.This field is either in a database table or view or a global variable.
As a table or view field it belongs to a
Groupobject.- __init__(identifier: str | None = None, datatype: ~ll.vsql.DataType = <ll.vsql.DataType.NULL: 'null'>, fieldsql: str | None = None, joinsql: str | None = None, refgroup: ~ll.vsql.Group | None = None)[source]
Create a
Fieldinstance.Argument are:
identifierThe UL4 identifier of the field
datatypeThe vSQL datatype of the field
fieldsqlThe SQL expression for that fields. This should include
{a}as a placeholder for the table alias.joinsqlIf this field is a foreign key to another table,
joinsqlis the join condition. This should include{m}and{d}placeholder for the table aliases of the master table (i.e. the one where this field is in) and the detail table (i.e. the one that will be joined).refgroupThe
Groupobject that represents the target table.
- class ll.vsql.Group[source]
Bases:
ReprA
Groupobject describes a group of database fields.These fields are part of a database table or view and are instances of
Field.
- class ll.vsql.Query[source]
Bases:
ReprA
Queryobject can be used to build an SQL query using vSQL expressions.- __init__(comment: str | None = None, **vars: Field)[source]
Create a new empty
Queryobject.Arguments are:
commentstrorNoneA comment that will be included in the generated SQL.
varsFieldThese are the top level variables that will be availabe for vSQL expressions added to this query. The argument name is the name of the variable. The argument value is a
Fieldobject that describes this variable.In most cases this
Fieldobject is a foreign key to another table, so it has ajoinsqland arefgroup.
- register_vsql(identifier: str) str | None[source]
Registers the field identifier
identifier.identifiermust belong to one of the fields passed to the constructor and it should reference a table.register_vsql()will then make sure that this referenced table will be added to the “from” list, even if it is never referenced explicitely in any of the “from” and “where” clauses.
- select_vsql(expr: str, alias=None) Query[source]
Add the vSQL expression
exprto the list of expression to select.aliascan be used to give the expression a column alias.Note that this compiles
exprand adds the resulting SQL. To add an SQL expression directly useselect_sql()instead.
- select_sql(expr: str, alias=None, comment=None) Query[source]
Add the SQL expression
exprto the list of expression to select.aliascan be used to give the expression a column alias.commentcan be used to give the column a comment in the select list.Note that that adds
exprdirectly as “raw” SQL. To add a vSQL expression useselect_vsql()instead.
- from_sql(tablename, alias, comment=None) Query[source]
Add a table to the list of tables to select from.
This adds the table in “raw” SQL form.
There’s no need to add to the “from” list in vSQL form, since this is done automatically in
select_vsql(),where_vsql()ororderby_vsql().
- where_vsql(expr: str) Query[source]
Add vSQL condition
exprto thewhereclause.Note that this compiles
exprand add the resulting SQL. To add an SQL expression directly usewhere_sql()instead.If
exprdoesn’t have the datatypeBOOLit will be automatically converted toBOOL.
- where_sql(expr: str, comment: str | None = None) Query[source]
Add vSQL condition
exprto thewhereclause.Note that that adds
exprdirectly as “raw” SQL. To add a vSQL expression usewhere_vsql()instead.
- orderby_vsql(expr: str, direction: None | Literal['asc', 'desc'] = None, nulls: None | Literal['first', 'last'] = None) Query[source]
Add the “order by” vSQL expression
exprto this query.“order by” specifications will be output in the query in the order they have been added.
Note that this compiles
exprand adds the resulting SQL. To add an SQL expression directly useorderby_sql()instead.Argument are:
exprstrvSQL expression to be sorted by
directionNone,"asc"or"desc"Sort in ascending order (
"asc") or descending order ("desc").The default
Noneadds neitherascnordesc(which is equivalent toasc).nullsNone,"first"or"last"Output
nullvalues first or last.The default
Noneadds neithernulls firstnornulls last.
Example:
>>> from ll import vsql >>> q = vsql.Query("Example query", user=la.User.vsqlfield()) >>> q.select_vsql("user.email") \ ... .orderby_vsql("user.firstname", "asc") \ ... .orderby_vsql("user.surname", "desc") >>> print(q.sqlsource()) /* Example query */ select t1.ide_account /* user.email */ from identity t1 /* user */ where livingapi_pkg.global_user = t1.ide_id(+) /* user */ order by t1.ide_firstname /* user.firstname */ asc nulls first, t1.ide_surname /* user.surname */ desc nulls last
- orderby_sql(expr: str, direction: None | Literal['asc', 'desc'] = None, nulls: None | Literal['first', 'last'] = None) Query[source]
Add the “order by” SQL expression
exprto this query.“order by” specifications will be output in the query in the order they have been added.
Note that that adds
exprdirectly as “raw” SQL. To add a vSQL expression useselect_vsql()instead.For an explanation of the rest of the arguments see
select_vsql().
- offset(offset: int | None) Query[source]
Use
offsetas the offset value.This offset specifies how mnay records to skip before returing the first one. The default 0 or None doesn’t skip any records.
- class ll.vsql.Rule[source]
Bases:
ReprRuleis used to store a type specific vSQL grammar rule.I.e. one rule object stores the information that:
there’s and addition operator;
that adds two
INTvalues;with a result of type
INT;and the SQL code to generate for that operation.
For more information see
AST.add_rules().
- class ll.vsql.AST[source]
Bases:
ReprBase class of all vSQL abstract syntax tree node types.
The following class attribute is used:
- title: str
Contains a human readable name for the AST type.
Instance attributes are:
- nodetype: NodeType
Type of the node. There’s a one-to-one correspondence between
ASTsubclasses andNodeTypevalues (except for intermediate classes likeBinaryAST)
- nodevalue: str
The node value is an instance attribute that represents a string that isn’t represented by any child node. E.g. the values of constants or the names of functions, methods and attributes. Will be overwritten by properties in subclasses.
- datatype: DataType | None
The datatype is an instance attribute that represents the datatype of the expression.
If the datatype can’t be determined because of errors datatype will be None.
- __init__(*content: AST | str)[source]
Create a new
ASTnode from its content.contentis a mix ofstrobjects containing the UL4 source and childASTnodes.Normally the user doesn’t call
__init__()directly, but usesmake()to create the appropriateASTnode from child nodes.For example a function call to the function
datecould be created like this:FuncAST( "date", "(", IntAST("2000", 2000), ", ", IntAST("2", 2), ", ", IntAST("29", 29), ")", )
but more conveniently like this:
FuncAST.make( "date", ConstAST.make(2000), ConstAST.make(2), ConstAST.make(29), )
- abstractmethod classmethod make() AST[source]
Create an instance of this AST class from its child AST nodes.
This method is abstract and is overwritten in each subclass.
This is a very low level way of creating vSQL expressions.
For example a vSQL expression for
"foo".lower() + "bar".upper()can be constructed like this:vsql.AddAST.make( vsql.MethAST.make( vsql.StrAST.make("foo"), "lower", ), vsql.MethAST.make( vsql.StrAST.make("bar"), "upper", ), )
- classmethod fromsource(source: str, **vars: Field) AST[source]
Create a vSQL expression from it source code.
For example
"foo".lower() + "bar".upper()can be compiled like this:vsql.AST.fromsource("'foo'.lower() + 'bar'.upper()")
varscontains the “root” variables that can be referenced in the vSQL expression.
- fieldrefs() Generator[FieldRefAST, None, None][source]
Return all
FieldRefASTobjects in thisAST.This is a generator.
- classmethod all_types() Generator[Type[AST], None, None][source]
Return this class and all subclasses.
This is a generator.
- classmethod all_rules() Generator[Rule, None, None][source]
Return all grammar rules of this class and all its subclasses.
This is a generator.
- classmethod add_rules(spec: str, source: str) None[source]
Register new syntax rules for this AST class.
These rules are used for type checking and type inference and for converting the vSQL AST into SQL source code.
The arguments
specandsourcehave the following meaning:specspecspecifies the allowed combinations of operand types and the resulting type. It consists of the following:- Upper case words
These specify types (e.g.
INTorSTR; for a list of allowed values seeDataType). Also allowed are:Tfollowed by an integer, this is used to refer to another type in the spec anda combination of several types joined with
_. This is a union type, i.e. any of the types in the combination are allowed.
- Lower case words
They specify the names of functions, methods or attributes
- Any sequence of whitespace or other non-word characters
They are ignored, but can be used to separate types and names and to make the rule clearer.
The first word in the rule always is the result type.
Examples:
INT <- BOOL + BOOLAdding this rule to
AddASTspecifies that the typesBOOLandBOOLcan be added and the resulting type isINT. Note that using+is only syntactic sugar. This rule could also have been written asINT BOOL BOOLor even asINT?????BOOL#$%^&*BOOL.INT <- BOOL_INT + BOOL_INTThis is equivalent to the four rules:
INT <- BOOL + BOOL,INT <- INT + BOOL,INT <- BOOL + INTandINT <- INT + INT.T1 <- BOOL_INT + T1This is equivalent to the two rules
BOOL <- BOOL + BOOLandINT <- INT + INT.
Note that each rule will only be registered once. So the following code:
AddAST.add_rules( "INT <- BOOL_INT + BOOL_INT", "..." ) AddAST.add_rules( "NUMBER <- BOOL_INT_NUMBER + BOOL_INT_NUMBER", "..." )
will register the rule
INT <- BOOL + BOOL, but notNUMBER <- BOOL + BOOLsince the first call already registered a rule for the signatureBOOL BOOL.sourcesourcespecifies the SQL source that will be generated for this expression. Two types of placeholders are supported:{s1}means “embed the source code of the first operand in this spot” (and{s2}etc. accordingly) and{t1}embeds the type name (in lowercase) in this spot (and{t2}etc. accordingly).Example 1:
AttrAST.add_rules( f"INT <- DATE.year", "extract(year from {s1})" )
This specifies that a
DATEvalue has an attributeyearand that for such a valuevaluethe generated SQL source code will be:extract(year from value)
Example 2:
EQAST.add_rules( f"BOOL <- STR_CLOB == STR_CLOB", "vsqlimpl_pkg.eq_{t1}_{t2}({s1}, {s2})" )
This registers four rules for equality comparison between
STRandCLOBobjects. The generated SQL source code for comparisons betweenSTRandSTRwill bevsqlimpl_pkg.eq_str_str(value1, value2)
and for
CLOB/CLOBcomparison it will bevsqlimpl_pkg.eq_clob_clob(value1, value2)
- validate() None[source]
Validate the content of this AST node.
If this node turns out to be invalid
validate()will set the attributedatatypetoNoneanderrorto the appropriateErrorvalue.If this node turns out to be valid,
validate()will set the attributeerrortoNoneanddatatypeto the resulting data type of this node.
- check_valid(context: str | None = None) None[source]
Makes sure that
selfis valid.If
selfis invalid an appropriate exception will be raised.contextshould describe the context in which the expression is used. E.g."select"when used inQuery.select_vsql()or"where"when used inQuery.where_vsql().
- walknodes() Generator[AST, None, None][source]
Return the all child AST nodes of this node (recursively).
- walkpaths() Generator[list[AST], None, None][source]
Return an iterator for traversing the syntax tree rooted at
self.Items produced by the iterator paths are lists containing the path from the root
ASTobject toself.Note that the iterator will always produce the same list object that will be changed during the iteration. If you want to keep the value produced during the iteration, you have to make copies.
- class ll.vsql.BoolAST[source]
Bases:
_ConstWithValueASTA boolean constant (i.e.
TrueorFalse).
- class ll.vsql.IntAST[source]
Bases:
_ConstWithValueASTAn integer constant.
- class ll.vsql.NumberAST[source]
Bases:
_ConstWithValueASTA number constant (containing a decimal point).
- class ll.vsql.StrAST[source]
Bases:
_ConstWithValueASTA string constant.
- class ll.vsql.CLOBAST[source]
Bases:
_ConstWithValueASTA CLOB constant.
This normally will not be created by the Python implementation
- class ll.vsql.ColorAST[source]
Bases:
_ConstWithValueASTA color constant (e.g.
#fff).
- class ll.vsql.DateAST[source]
Bases:
_ConstWithValueASTA date constant (e.g.
@(2000-02-29)).
- class ll.vsql.DateTimeAST[source]
Bases:
_ConstWithValueASTA datetime constant (e.g.
@(2000-02-29T12:34:56)).
- class ll.vsql.ListAST[source]
Bases:
_SeqASTA list constant.
For this to work the list may only contain items of “compatible” types, i.e. types that con be converted to a common type without losing information.
- class ll.vsql.SetAST[source]
Bases:
_SeqASTA set constant.
For this to work the set may only contain items of “compatible” types, i.e. types that can be converted to a common type without losing information.
- class ll.vsql.FieldRefAST[source]
Bases:
ASTReference to a field defined in the database.
- __init__(parent: FieldRefAST | None, identifier: str, field: Field | None, *content: AST | str)[source]
Create a
FieldRefobject.There are three possible scenarios with respect to
identifierandfield:field is not None and field.identifier == identifierIn this case we have a valid
Fieldthat describes a real field.field is not None and field.identifier != identifier and field.identifier == "*"In this case
fieldis theFieldobject for the generic typed request parameters. E.g. when the vSQL expression isparams.str.foothenfieldreferences theFieldforparams.str.*, sofield.identifier == "*" and identifier == "foo".field is NoneIn this case the field is unknown.
- class ll.vsql.BinaryAST[source]
Bases:
ASTBase class of all binary expressions (i.e. expressions with two operands).
- class ll.vsql.UnaryAST[source]
Bases:
ASTBase class of all unary expressions (i.e. expressions with one operand).
- class ll.vsql.JavaSource[source]
Bases:
objectA
JavaSourceobject combines the source code of a Java class that implements a vSQL AST type with the Python class that implements that AST type.It is used to update the vSQL syntax rules in the Java implemenation of vSQL.
- new_lines() Generator[str, None, None][source]
Return an iterator over the new Java source code lines that should replace the static initialization block inside the Java source file.
- save() None[source]
Resave the Java source code incorporating the new vSQL type info from the Python AST class.
- ll.vsql.oracle_sql_procedure() str[source]
Return the SQL statement for creating the procedure
VSQLGRAMMAR_MAKE.