History

This document describes the new features, bug fixes and changes in each version of XIST. For a description of how to update your code to each versions of XIST see Migration.

Changes in 5.77 (released 2024-11-13)

  • The code has been updated to use X | Y for union types in some type annotations, so Python 3.10 is required now.

  • ll.orasql has been updated to use oracledb.

  • Instead of clientinfo ll.orasql now automatically includes the running script in program. If you don’t want that (for example when the path includes characters that are not allowed, so you get DPY-3029), pass program=None (or your own custom value).

  • The SQL output by ll.orasql.Job.dropsql() now includes the argument p_force=>true, so a job can be dropped even if it is currently running.

  • UL4 gained the type operator.itemgetter.

Changes in 5.76 (released 2024-07-08)

  • Fixed a bug in UL4 where calling the renders method during calls in a renderx tag resulted in the output being escaped twice.

    To fix this problem, the output infrastructure for UL4 (which was using generators before) has been switched to a stream API.

  • orafind now supports table and column names with “fancy” characters by quoting names when generating queries.

Changes in 5.75.1 (released 2024-04-11)

  • Added support for the HTML pseudo class :disabled to ll.xist.css.applystylesheets(). Now the following works:

    >>> from ll.xist.ns import html
>>> from ll.xist import css
>>> e = html.html(
...      html.style('''
...         button {color: blue;}
...         button:disabled {color: red;}
...         '''),
...      html.button('gurk', disabled=True)
...   )
...
>>> css.applystylesheets(e)
>>> e.string()
'<html><button disabled="disabled" style="color: red">gurk</button></html>'

Changes in 5.75 (released 2023-11-21)

  • In UL4 <?doc?> and <?note?> can now be nested, and otherwise can contain anything up to the matching <?end doc?> or <?end note?> tag, but only if the initial <?doc?> or <?note?> tag contains only whitespace (e.g. <?doc    ?> etc.). This makes it possible to have source code examples in <?doc?> or <?note?> tags.

  • Added a new class BoundTemplate to ll.ul4c: This is an UL4 template bound to an object.

    Calling or rendering a BoundTemplate instance passes the object to which the template is bound as the first positional argument.

  • Add attribute namespace to UL4 templates.

  • Add method timestamp to UL4 type datetime.

  • The UL4 function getattr now returns an UndefinedKey object when accessing an undefined attribute and no default value is passed.

  • The result of calling dir() on an object in UL4 can now be customized by implementing ul4_dir().

  • Updated UL4 and UL4ON test infrastructure to work around problems with gradle.

  • Replace a check for the ElementTree method getchildren() with a check for findall() in xml2xsc.iterpath() since getchildren() has been removed in Python 3.9.

  • Copied the implementation of cgi.parse_header() to ll.misc since cgi.parse_header() will be deprecated in Python 3.13. (cgi.parse_header() is used by ll.url, ll.xist.ns.html and ll.xist.ns.jsp).

Changes in 5.74 (released 2023-03-01)

  • Fixed handling of the cond attribute in the PySQL command commit and rollback.

  • Bumped required cssutils version to at least 2.6.0.

  • ll.xist.css now ignores the CSS pseudo classes :valid and :invalid.

  • Since the Java and Javascript versions of UL4 now supports format(float) this is now checked by the tests.

  • The UL4 function format(float) now honors the specified language and uses the correct decimal separator.

  • Running the UL4 tests for Java now is done via Gradle, so the tests no longer require Java UL4 and all its dependencies on the CLASSPATH. (Instead a full Java UL4 checkout is required in ~/checkouts/LivingLogic.Java.ul4.)

Changes in 5.73.2 (released 2022-08-16)

Changes in 5.73.1 (released 2022-08-10)

  • Fixed handling of the PySQL variable connection on connect and disconnect calls.

Changes in 5.73 (released 2022-08-10)

  • ll.pysql now supports Postgres. To connect to a Postgres database pass a connectstring to connect starting with postgres:, for example:

    connect("postgres:host=localhost dbname=test user=me password=secret")

    This will create a Postgres database connection via:

    psycopg.connect(
   "host=localhost dbname=test user=me password=secret",
   cursor_factory=extras.DictCursor
)

    All other connectstrings will be interpreted as Oracle connectstrings. An Oracle connectstring may start with the prefix oracle: which will be stripped off, before passing it to cx_Oracle.connect() or ll.orasql.connect().

    Note that Postgres currently doesn’t support the drop_types command.

  • Some PySQL commands have been renamed: resetsequence to reset_sequence, checkerrors to check_errors, raiseexceptions to raise_exceptions, pushraiseexceptions to push_raise_exceptions and popraiseexceptions to pop_raise_exceptions.

  • The PySQL commands procedure and sql have an additional argument argtypes that can be used to add casts to the parameter values in the call to convert the value to the proper Postgres datatype (to guide Postgres to find the correct overloaded version of the procedure).

  • When a var object is passed a second time in PySQL, now instead of the variable’s value a proper variable object will be passed to the procedure or sql call. This means if the variable gets changed by the call, the new value will be picked up by the local variable.

    If you want to pass the variable’s value instead as a simple IN parameter, simply pass the local variable instead.

  • The argument raiseexceptions to various PySQL commands has been renamed to raise_exceptions.

Changes in 5.72 (released 2022-08-04)

  • ll.ul4c.TextAST objects now always store the final text string instead of slicing it from the source code on every call.

  • UL4 AST classes store source offsets in the UL4ON dump now as two ints instead of a slice object. The Java and Javascript implementations directly use these two ints instead of converting them to a slice object. This reduces memory usage in the Java version. However the Python version still stores slice objects internally.

Changes in 5.71 (released 2022-07-08)

  • Fixed ll.xist.xfind.selector() when running under Python 3.9.

  • Restored support for ll.xist.xsc._Node_Meta.__or__() for XIST classes under Python 3.9.

  • ll.xist.xfind.Selector now implements the reflected operators too. This reenables certain argument combinations as ll.xist.xsc._Node_Meta.__or__() isn’t available on Python 3.10.

  • ll.xist.xfind.NotCombinator now converts its argument to a Selector object, instead of expecting it to already be one.

  • Publishing of <script> elements now properly outputs the elements content as CDATA unless in XML mode. I.e.

    html.script("&").string()

    will now output <script>&</script> and

    html.script("&").string(xhtml=2)

    outputs <script>&amp;</script> as before.

Changes in 5.70 (released 2022-03-11)

  • Fixed a bug in the PySQL command commit: Removed the useless argument sql.

  • Fixed a bug in the method ll.orasql.Connection.getobject() (which we only keep for backwards compatibility).

  • Fixed a bug in parsing UL4 source code: UL4 would treat a tag name that starts with a valid prefix as a valid tag name, although it shouldn’t (i.e. it treated <?printe?> as <?print e?>).

  • Added new UL4 AST classes: RenderOrPrintAST, RenderOrPrintXAST, RenderXOrPrintAST and RenderXOrPrintXAST.

Changes in 5.69 (released 2021-11-17)

Changes in 5.68.1 (released 2021-09-23)

  • Fixed a bug in ll.pysql that resulted in the local variable connection being wrapped in a tuple.

Changes in 5.68 (released 2021-08-04)

  • UL4 templates now support the <?ignore?> tag. It can be used to “comment out” template code. <?ignore?>/<?end ignore?> tags nest.

  • Added the following three methods to ll.orasql.Table:

    • ll.orasql.Table.fks() returns all foreign keys for the table;

    • ll.orasql.Table.uniques() returns all unique constraints for the table;

    • ll.orasql.Table.checks() returns all check constraints for the table.

  • Fixed scoping problems in literal Python blocks in PySQL scripts: List comprehension were not able to access local variables.

  • Removed the ancient XIST namespace modules ll.xist.ns.kid and ll.xist.ns.ihtml.

Changes in 5.67.2 (released 2021-06-30)

  • Fixed handling of values for the --define/--D option of ll.pysql.

Changes in 5.67.1 (released 2021-06-28)

  • Fixed a typo in the tag external when logging exceptions in Sisyphus jobs.

Changes in 5.67 (released 2021-06-25)

  • Added the options --sentry_environment, --sentry_release and --sentry_debug to Sisyphus jobs.

  • Sisyphus jobs now warning when the modulde sentry_sdk can not be imported.

  • For Sisyphus jobs it’s now possible to log to the ll.sisyphus.EMailLogger, ll.sisyphus.MattermostLogger and ll.sisyphus.SentryLogger by using the tag external.

  • The Sispyhus log message for setting up the Sentry logging is now a delayed message.

  • The method ll.orasql.Connection.getobject() has been renamed to ll.orasql.Connection.object_named().

  • The method ll.orasql.Connection.objects_name() has been added that returns all database objects with the specified name.

Changes in 5.66.1 (released 2021-06-24)

  • When a sisyphus job logs to Sentry, flush messages after logging them to make sure that they arrive even in forking mode.

Changes in 5.66 (released 2021-06-15)

  • UL4 now use functions and methods with positional-only parameters, so Python 3.8 is required now.

  • UL4 functions and methods have been updated to use positional-only or keyword-only arguments to match the signature of the corresponding Python function/method.

  • Some functions now use positional-only arguments:

    • ll.misc.item(iterable, index, /, default=None)

    • ll.misc.first(iterable, /, default=None)

    • ll.misc.last(iterable, /, default=None)

    • ll.misc.count(iterable, /)

    • ll.misc.isfirst(iterable, /)

    • ll.misc.islast(iterable, /)

    • ll.misc.isfirstlast(iterable, /)

    • ll.misc.monthdelta.__init__(self, months=0, /)

    • ll.ul4on.dumps(obj, /, indent)

    • ll.ul4on.dump(obj, /, stream, indent)

    • ll.ul4on.load(stream, /, registry=None)

    • ll.ul4on.loads(dump, /, registry=None)

    • ll.ul4on.loadclob(clob, /, bufsize=1024*1024, registry=None)

    as well as the UL4 function enumfl(iterable, /).

  • Subclasses of ll.ul4c.AST have been renamed so that their name matches the name of the corresponding class in the Java implementation. (For example ll.ul4c.Add has been renamed to ll.ul4c.AddAST).

  • The UL4 function type() now returns “type objects” instead of simple strings. A type object can be used for checking whether an object is an instance of a certain type (via the newly introduced function isinstance()). Some type objects can also be used to create new instances of that type (by calling the type object).

  • The following builtin UL4 functions are now callable type objects: bool, int, float, str, date, datetime, timedelta, monthdelta, list, set, dict and color.Color.

  • The following modules have been added to the builtin UL4 objects: ul4 contains all UL4 AST classes (ul4.Template is callable to create an UL4 template from source), ul4on contains the functions dumps() and loads() and the types Encoder and Decoder, operator contains the type attrgetter and math contains the constants e, pi and tau as well as the functions cos(), sin(), tan(), sqrt() and isclose(). color contains the type Color and the functions css and mix.

  • Naming of attributes that are used by UL4 to implement UL4 functionality has been made more uniform. This affects the following attributes: The methods ul4_getattr(), ul4_setattr() and ul4_hasattr() for implementing object attribute access from UL4; the methods ul4_call(), ul4_render() and ul4_renders() for making objects callable or renderable from UL4; the class attribute ul4_attrs for exposing a number of readonly attributes to UL4 and the attribute ul4_context that is used for marking a callable as needing the context as an argument in the call.

  • Now the ul4_attr attribute of objects gets honored in the implementation of the dir() function in UL4.

  • Support for using custom tag delimiters for UL4 templates has been removed, i.e. now the tag delimiters will always be <? and ?>.

  • ll.ul4on.Decoder gained a new method forget_persistent_object().

  • ll.sisyphus.Jobs can now log to Sentry.

  • ll.sisyphus.Tasks constructor and the method ll.sisyphus.Job.task now take arbitrary additional keyword arguments. Those will be passed as additional breadcrumb data when logging to Sentry.

  • ll.sisyphus.Job.tasks() now takes an additional argument data that is responsible for returning additional data for the task.

  • The following methods of ll.color.Color have been renamed: abslum() to abslight() and rellum() to rellight().

  • The following methods have been added to ll.color.Color: hue(), light(), sat(), withhue(), withsat(), withlum(), abslum(), rellum() and invert(). They have also been made available to UL4. The color method combine() and the functions ll.color.css() and ll.color.mix() are now also available to UL4.

Changes in 5.65 (released 2021-01-13)

  • ll.ul4on now supports “persistent” objects, i.e. objects that can be uniquely identified across several unrelated decoding calls. Loading a persistent object that has been loaded before will reinitialize the existing object instead of creating a new one.

  • In sisyphus jobs, when the value of the nextrun parameter was a datetime.timedelta object, it was interpreted relative to the start of the run, not relative to now. This has been fixed.

  • The modules ll._xml_codec and ll.xist.sgmlop have been updated to directly support and produce PEP 393 style unicode strings. (ll._misc already supported them).

  • A method date has been added to date and datetime objects in UL4 templates.

Changes in 5.64 (released 2020-10-30)

  • It is now possible to specify a custom port for ssh URLs.

  • A second URL scheme ssh-nocheck has been added for ssh URLs. Using ssh-nocheck disables the host key check when establishing the ssh connection.

  • With these changes it is now possible to do the following:

    >>> from ll import url
>>> u = url.URL("ssh-nocheck://www.example.org:2222/~/foo.txt")
>>> with url.Context():
...   data = u.open("rb").read()

Changes in 5.63.1 (released 2020-10-26)

Changes in 5.63 (released 2020-09-08)

  • All PySQL commands now support the argument cond.

  • Added a PySQL command constraint_exists that checks the existance of database constraints.

Changes in 5.62 (released 2020-07-13)

  • Update HTML for the documentation to use our own (RTD based) theme instead of using and overwriting the RTD theme.

Changes in 5.61.2 (released 2020-07-09)

  • Fixed a regression in ll.sisyphus from 5.61.1: Stay in delayed logs mode after a successful job run.

Changes in 5.61.1 (released 2020-07-09)

  • Fixed handling of the --exit_on_error option in ll.sisyphus.

  • Rewrote handling of delayed logs in ll.sisyphus to work better in forking mode.

Changes in 5.61 (released 2020-07-07)

  • Updated handling of custom data types when calling functions or procedures in ll.orasql to work with cx_Oracle 8.0.

  • Reordered table comments, column comments and the primary key of a table in the output of ll.orasql.Connection.objects() so that these objects have the same order as in ll.orasql.Table.referencedby().

Changes in 5.60 (released 2020-07-03)

  • The handling of delayed logs and uneventful runs in ll.sisyphus has been changed: “Delayed logs” mode is now always active. If only delayed log messages are output they will never be written to the logfiles. If a job run is uneventful (i.e. execute() returns None) no log messages will be written. If the job run is successful only the job result will be written.

    The option and the class/instance attribute delaylogs no longer exist.

  • Added a new option and class/instance attribute exit_on_error which will stop job execution in repeat mode when an exception occurs.

Changes in 5.59 (released 2020-06-30)

  • ll.orasql now requires cx_Oracle 8.0.

  • ll.orasql.Connection.objects() now outputs TableComment objects too.

  • Fixed ll.orasql.Table.comment() when the table was owned by a different user.

Changes in 5.58 (released 2020-06-12)

  • For running healthchecks for sisyphus jobs it’s no longer neccessary (or even allowed) to implement the healthcheck() method. Instead the job writes a healthfile at the end of each run, and the age and content of this file will be used to determine the health of the job. The option --maxhealthcheckage can be used to configure the maximum allowed age.

  • Logging to emails was broken when sisyphus jobs were running in fork mode (the default): The child process was collecting log messages for the email, but the parent process didn’t, so it never sent an email. This has been fixed now: Both processes write log messages to files, and those will be used after the job run to create the email.

  • Now links will be created for every possible result status of a job run. So it’s immediate clear when the last successful job run was, when the last job run failed with an exception, was canceled or timed out.

  • The filenames for log files can no longer be changed via options or job attributes, instead one of the following methods must be overwritten:

    • basedir()

    • logfilename()

    • currentloglinkname()

    • lastsuccessfulloglinkname()

    • lastfailedloglinkname()

    • lastinterruptedloglinkname()

    • lasttimeoutloglinkname()

    • healthfilename()

    • emailfilename()

    Those methods must return an absolute path as a pathlib.Path object.

Changes in 5.57 (released 2020-04-14)

  • Added a “delayed logs” mode to ll.sisyphus. This makes it possible to delay output of any log messages until something interesting happens. When nothing interesting happens, log messages will be thrown away.

  • Use pathlib internally for handling log file names in ll.sisyphus.

  • When a ll.sisyphus job compresses log files the compressed log file now retains the modification timestamp of the original log file.

  • The API for ll.ul4on.Encoder and ll.ul4on.Decoder has been updated to support multiple calls for encoding/decoding an UL4ON dump to multiple strings or streams that nonetheless keep the state for the encoding/decoding machinery.

  • UL4ON functionality is now available to UL4 templates in a ul4on “module”. This module provides the functions/types loads, dumps, Decoder and Encoder.

  • The parameter string for the UL4 function fromul4on has been renamed to dump.

Changes in 5.56 (released 2019-12-12)

  • ll.orasql.Comment has been renamed to ll.orasql.ColumnComment.

  • Added a class ll.orasql.TableComment for table comments.

  • Added a method ll.orasql.Table.comment() that returns the ll.orasql.TableComment object for this table.

  • UL4 templates now support global variables. To be able to pass global variables to UL4 templates the following methods have been added to ll.ul4c.Template: render_with_globals(), renders_with_globals() and call_with_globals().

Changes in 5.55 (released 2019-11-11)

  • Added an option --ignoreerrors to orareindex.

  • UL4 dictionaries now have a method pop().

  • Added an UL4 function scrypt implementing the scrypt hashing function (see https://en.wikipedia.org/wiki/Scrypt).

  • Added a new method ll.orasql.Table.compression() that returns the table compression (None, "BASIC" or "ADVANCED").

  • Added a new method ll.orasql.Column.compression() that returns the column compression for LOB columns (None, "LOW", "MEDIUM" or "HIGH").

  • ll.orasql.Table.createsql() now can handle table and LOB column compression.

  • Added a method loadcontentitems() to the class ll.ul4on.Decoder which can be used to load the content of an object as (key, value) pairs.

Changes in 5.54.1 (released 2019-10-24)

  • Fixed wrong HTTP header when posting sisyphus log entries to Mattermost.

Changes in 5.54 (released 2019-10-24)

  • The tab width used by ll.xist.ns.html.astext() is now configurable and long words will no longer be broken across multiple lines. This should prevent long URLs from being broken.

  • ll.misc.sysinfo now exposes its attributes to UL4.

  • ll.sisyphus log entries can now be logged to a Mattermost chat channel.

Changes in 5.53 (released 2019-09-30)

  • Fixed a bug in the handling of users and job classes (i.e. objects that don’t have an owner) in ll.orasql.OracleURLConnection.walk().

  • Added an option --healthcheck to ll.sispyhus jobs: Starting a job with this option runs a separate method healthcheck() that is used to check that the job is doing what it’s supposed to be doing.

Changes in 5.52.1 (released 2019-09-05)

  • Fixed handling of indentation when an UL4 <?renderblock?> contains a <?render?> call.

Changes in 5.52 (released 2019-07-29)

  • The method getobject() for ll.orasql.Synonym has been renamed to object().

  • A new method ll.orasql.Privilege.object() has been added. This method returns the object for which the ll.orasql.Privilege grants a privilege. E.g. if the Privilege p grant the SELECT privilege on a table, p.object() will return that table.

  • A new method ll.orasql.OwnedSchemaObject.synonyms() has been added. This generater yields all the synonyms for the object it is called on.

  • A new method ll.orasql.OwnedSchemaObject.privileges() has been added. This generater yields all the privileges for the object it is called on.

  • A new method ll.orasql.Connection.synonyms() has been added.

  • ll.orasql.Synonym.createsql() now omits the schema name from the name for the object if it’s the current user.

Changes in 5.51 (released 2019-07-26)

  • ll.orasql.Synonym.names() and ll.orasql.Synonym.objects() can now filter on the owner of the object (i.e. the object for which the Synonym is a synonym) via the new parameter object_owner.

  • Fixed the repr output of UL4 dictionary comprehensions.

Changes in 5.50 (released 2019-07-16)

  • There’s a new option -a/--ascii for running PySQL scripts: With this PySQL will not use unicode characters for drawing fancy boxes.

  • Fixed a bug in the filename handling of PySQL, so that showing source in stacktraces works again: as we’re changing directories now, using relative paths no longer worked.

  • PySQL no longer uses exception chaining for displaying the location and the include chain of an error, as this is now part of the normal stacktrace anyway.

  • Fixed logic for showing line numbers for locations in PySQL scripts.

  • Fixed a bug in the PySQL command resetsequence.

  • Added classes ll.misc.Enum and ll.misc.IntEnum that are subclasses of enum.Enum and enum.IntEnum, but show the module and fully qualified class name in the repr() output for classes and instances.

Changes in 5.49 (released 2019-07-04)

  • Privileges returned by ll.orasql.Privilege.objects() will now have a stable sort order.

Changes in 5.48 (released 2019-07-03)

  • Filename printed by a PySQL script will now always be relative to the current directory at the start of the script.

  • Fixed a bug in the filename handling of the PySQL command file.

Changes in 5.47 (released 2019-07-01)

  • Include commands in PySQL scripts now actually change the current directory so that literal Python blocks execute with the current directory set to the directory of the PySQL file containing the Python block.

  • The PySQL commands procedure and sql now report new variable values.

Changes in 5.46 (released 2019-06-26)

  • The method ll.scripts.rul4.Globals.log() now supports the keywords arguments sep, end and flush with the same meaning as for print().

  • Exception chaining in ll.scripts.rul4.Globals.error() has been fixed.

  • For ll.xist.ns.html.astext() default styles have been added for em, strong and i.

  • ll.xist.ns.html.astext() now honors all styles passed as keyword arguments not just those for h1, h2, h3, h4, h5, h6, dl, dt, dd, ol, ol_li, ul, ul_li, pre, blockquote, div, p, hr, address, th, td, b, u and code.

  • ll.xist.ns.html.astext() now supports callables for the prefix and suffix style parameter. If a callable is passed it will be called with the node in question and the resulting string is used as the prefix/suffix. For example it’s possible to output links in Markdown syntax like this:

    >>> from ll.xist.ns import html
>>> e = html.p(
...   "We ",
...   html.em("love"),
...   " ",
...   html.a("Python", href="http://www.python.org/"),
...   "!"
... )
>>> html.astext(
...   e,
...   a=dict(
...      display="inline",
...      prefix="(",
...      suffix=lambda n: f")[{n.attrs.href}]"
...   )
... )
'We *love* (Python)[http://www.python.org/]!'

Changes in 5.45 (released 2019-06-24)

  • UL4 AST nodes for blocks now have additional attributes startpos and stoppos. startpos is the position of the start tag and stoppos is the position of the end tag. The attributes line and col have been renamed to startline and startcol and attributes stopline and stopcol have been added.

    Furthermore two attributes startsource and stopsource have been added. They return the source code of the start tag and the end tag. So for example for the loop <?for i in range(10)?><?print i?><?end for?> the startsource is <?for i in range(10)?> and the stopsource is <?end for?> (and source is <?for i in range(10)?><?print i?><?end for?>).

    Additionally attributes startsourceprefix, startsourcesuffix, stopsourceprefix and stopsourcesuffix have been added.

    In exception messages startsource is now used as the exception location. This means when a for loop iterates over something that is not iteratable the location marked will now be the loop start tag instead of the complete loop.

    (For non-block nodes startpos is the same as pos, startsource is the same as source, startsourceprefix is the same as sourceprefix and startsourcesuffix is the same as sourcesuffix.)

Changes in 5.44 (released 2019-06-07)

  • ll.orasql.Connection.objects() now outputs Job objects too. Since Oracle provides no dependency information about jobs, Job objects will always be output last (in “create” mode; in “drop” mode they will be output first).

  • ll.orasql.Job.references() will now yield the appropriate ll.orasql.JobClass object (if the job class isn’t a system job class).

  • ll.orasql.JobClass.referencedby() will now yield all ll.orasql.Job objects that use this job class.

  • The owner argument for various ll.orasql methods now supports passing a set or tuple of owner names.

  • PySQL scripts now can contains PySQL commands in “function call form”, i.e. checkerrors() instead of {'type': 'checkerrors'}.

  • PySQL scripts can now contains literal Python source code (between lines with #>>> and #<<<, e.g.:

    #>>>
``cursor`` .cursor()
cursor.execute("drop user foo cascade")
#<<<

  • Comments in PySQL scripts are supported now (via lines starting with #).

  • Since PySQL scripts can open their own database connections the connectstring argument for the pysql script is now optional.

  • The PySQL command compileall has been removed. This same effect can simply be achieved by calling utl_recomp.recomp_parallel() or dbms_utility.compile_schema().

  • Added several new PySQL commands: commit and rollback, drop_types, user_exists, object_exists and env.

  • The --commit argument for the pysql script (with the options record, once and never) has been replaced with a flag option --rollback. Automatically committing after every record is no longer available.

  • The PySQL terminator comment (-- @@@) can now no longer be specified via a command line option.

  • The -v/--verbose option for ll.pysql now supports new output modes (file and log) and full mode now outputs much more information.

  • The URL methods owner() and group() now will return the uid or gid respectively when the user or group name can’t be determined instead of raising a KeyError.

  • Fixed SQL statement for dropping ll.orasql.Job objects.

Changes in 5.43 (released 2019-05-07)

Changes in 5.42.1 (released 2019-04-29)

  • Fixed a bug in ll.orasql.OracleURLConnection._walk() to support orasql objects that don’t have an owner (i.e. User and JobClass).

  • Simplified clean up logic for sisyphus jobs (which makes the new “delete log files for uneventful runs” logic work on Windows).

Changes in 5.42 (released 2019-04-26)

  • By returning None from the method ll.sisyphus.Job.execute() a sisyphus job can now report that the job run was “uneventful” (i.e. the job had nothing to do) and that the log file can be deleted immediately.

Changes in 5.41 (released 2019-03-29)

  • Added a script oracycles that can find cyclic foreign key references in an Oracle database schema.

Changes in 5.40.2 (released 2019-03-26)

  • The jobname for ll.orasql.Job objects now no longer includes the owner name.

Changes in 5.40.1 (released 2019-03-25)

  • Fixed a bug in ll.orasql for Comment objects.

Changes in 5.40 (released 2019-03-25)

  • ll.orasql now supports jobs and job classes via the new classes Job and JobClass.

  • The UL4 functions isfirst, islast and isfirstlast are now available as standalone Python functions in the ll.misc module.

Changes in 5.39 (released 2019-01-30)

  • ll.misc.SysInfo now uses platform.uname() instead of os.uname() for its host information, so this will work on Windows too. This also means ll.misc.SysInfo gained a new attribute host_processor which is provided by platform.uname(). Furthermore the user information on Windows now supports user_name and user_dir (all other user attribute are None).

  • ll.orasql objects that have source code (like Type, Procedure etc.) should now be able to better handle any duplicate spaces in its source code header introduced by Oracle 18.

Changes in 5.38 (released 2018-11-15)

  • Added the following attributes to the UL4 class AST: line (the line number in the template source code), col (the column number in the template source code), sourceprefix (a part of the templates source code before the source code of the AST node) and sourcesuffix (a part of the templates source code after the ASTs source code).

    These attributes are also accessible to UL4 templates.

Changes in 5.37.1 (released 2018-11-13)

  • Each UL4 AST node now has an attribute fullsource which is the full source of the outermost template.

  • Fixed source attribute of empty UL4 templates.

Changes in 5.37 (released 2018-11-08)

  • The chaining of UL4 exceptions has been inverted. This means that the exception that will get raised from the UL4 template is the original innermost exception. LocationError instances will be added to the __cause__ attribute to specify the exact location in the UL4 source. Note that this means that those LocationError instances won’t have a traceback added, as they will never be raised.

  • The structure of compiled UL4 templates has been simplified internally: Each AST instance has attributes template and pos that directly reference the template and the source code location of the AST node. The Tag objects are gone (they will only be used internally during compilation). Also AST nodes have gained a source property which returns the source code of the node itself.

Changes in 5.36 (released 2018-10-31)

  • As cx_Oracle provides its own class Object ll.orasqls class Object has been renamed to SchemaObject.

  • sisyphus jobs can now run even if os.fork() and fcntl are not available or signal.signal() doesn’t support SIGALRM (i.e. on Windows). In this case various features will be missing.

  • It is now possible to pass instances of cx_Oracle.Object as arguments when calling function or procedures in orasql.

Changes in 5.35 (released 2018-09-14)

  • UL4 now support both datetime and date objects. A function today has been added that returns the current day as a date object.

  • The UL4 method week for date objects now by default returns the ISO calendar week number. The day that should be considered the first day of the week and how many days the first week of the year has to have can be passed as parameters.

  • A new UL4 method calendar has been added that returns the ISO calendar year, the ISO calendar week and the weekday. Similar to week the day that should be considered the first day of the week and how many days the first week of the year has to have can be passed as parameters.

  • Speed up deserializing strings from UL4ON dumps.

  • ll.sisyphus now uses psutil to terminate all child processes when the maximum runtime is exceeded. If psutil is not available only the forked child process itself will be terminated as before.

  • Fixed a bug in ll.orasql.ForeignKey.refconstraint() for foreign keys that reference a table in another schema.

Changes in 5.34 (released 2018-06-03)

  • Renamed the class ll.xist.ns.html.script.Attrs.async because async is a keyword in Python 3.7.

  • XIST is Python 3.7 compatible now.

Changes in 5.33 (released 2018-05-15)

  • ll.orasql.PrimaryKey.columns() and ll.orasql.ForeignKey.columns() now yield ll.orasql.Column objects that link back to the db schema from which they originated (so it is possible to call methods on them without passing the database connection).

  • ll.orasql.Column has two new attributes: tablename is the name of the table that the column belongs to and columnname is the name of the column (without the table name).

  • Exceptions from ssh URLs are no longer checked whether they are from the correct module. Instead they’re always sent across the execnet channel, so that the receiving side has to deal with them.

  • Since UL4 relies on ordered dictionaries (i.e. ordinary dictionaries in Python 3.6) and Javascript doesn’t guarantee iteration order of objects and Spidermonkey doesn’t support sets and maps, testing UL4 with Spidermonkey has been dropped.

Changes in 5.32 (released 2018-02-20)

  • ll.orasql.Connection.objects() now makes sure that no objects from other schemas are returned when a specific schema owner is requested.

  • The default value for the owner parameter in various ll.orasql methods has changed from ALL to None (i.e. it now returns the objects from the current schema instead of all schemas).

Changes in 5.31 (released 2018-01-29)

  • The UL4ON decoder now has a new method loadcontent() that can be used to iteratively load the content of an object. This makes it possible to handle object dumps where the writing side dumped a different number of items than the reading side expects.

Changes in 5.30 (released 2018-01-17)

  • The new UL4 tag <?renderx?> works like <?render?>, but the output from the template will be XML escaped.

  • The new tag <?renderblocks?> is syntactic sugar for rendering a template and passing various other templates as keyword arguments, i.e. if we have a template:

    <?def page(head, body)?>
   <html>
      <head>
         <?render head()?>
      </head>
      <body>
         <?render body()?>
      </body>
   </html>
<?end def?>

    then:

    <?renderblocks page()?>
   <?def head?>
      <title>Foo</title>
   <?end def?>
   <?def body?>
      <h1>Bar!</h1>
   <?end def?>
<?end renderblocks?>

    is syntactic sugar for:

    <?def head?>
   <title>Foo</title>
<?end def?>
<?def body?>
   <h1>Bar!</h1>
<?end def?>

<?render page(head=head, body=body)?>

    except that with <?renderblocks?> the templates head and body will not leak into the surrounding namespace.

  • The new tag <?renderblock?> is similar to <?renderblocks?>. However the complete content of the <?renderblock?> call will be passed as the content keyword argument to the render call. I.e.:

    <?renderblock foo()?>
   bar
<?end renderblock?>

    is syntactic sugar for:

    <?def content?>
   bar
<?end def?>
<?render foo(content=content)?>

Changes in 5.29 (released 2017-11-29)

  • When an exception happens during decoding of an UL4ON stream the stack of types that is currently being decoded is included in the exception message now.

  • The Javascript implementations of UL4 and UL4ON are now tested against Node.js (in addition to V8 and Spidermonkey).

  • The UL4 string methods startswith and endswith now support lists of strings as an argument.

Changes in 5.28.2 (released 2017-08-03)

  • The character < is now escaped as \x3c in UL4ON dumps to help XSS prevention.

Changes in 5.28.1 (released 2017-08-02)

  • Fixed a bug in ll.sisyphus.Task.__str__().

  • The UL4 function asjson now escapes < as \u003c to help XSS prevention.

Changes in 5.28 (released 2017-08-01)

  • XIST requires Python 3.6 now.

  • As dicts are ordered in Python 3.6 the Attrs attribute xmlorder is gone. Attributes will always be serialized in the same order they have been parsed/created.

  • UL4 no longer tries to disguise objects as dictionaries. I.e. for objects with an ul4attrs class attribute the methods items, keys, values and get are no longer synthesized. This also means that len, list, item access and containment tests no longer work on objects.

  • New UL4 functions getattr, setattr, hasattr and dir have been added to work with attributes of objects.

  • Fixed an UL4ON bug: Strings containing line feeds can now be deserialized properly.

Changes in 5.27 (released 2017-03-21)

  • When deserializing UL4ON dumps it is now possible to pass in a custom type registry dictionary. This can be used to customize which objects get created for which type.

Changes in 5.26.1 (released 2017-03-03)

  • The fields of a unique constraint are now output in the correct order by ll.orasql.UniqueConstraint.createsql().

Changes in 5.26 (released 2017-02-28)

  • UL4 templates now support a new tag: <?doc?> may contain the documentation for the template and will by accessible in UL4 templates via the attribute doc.

  • The signature of UL4 templates is now available to UL4 templates as the signature attribute:

    <?def f(x=17, y=23)?>
   <?print x+y?>
<?end def?>
<?print f.signature?>

    will output

    (x=17, y=23)

Changes in 5.25.1 (released 2017-02-15)

  • Fixed a problem with the renders method of local UL4 templates. The local template didn’t see the variables from the surrounding scope.

Changes in 5.25 (released 2017-02-13)

  • UL4ON dumps can now contain UL4 templates in “source” format, i.e. the template will be compiled by the UL4ON decoder. This makes it possible to dump UL4 templates via PL/SQL code (see the LivingLogic.Oracle.ul4 project on GitHub for more info).

Changes in 5.24 (released 2017-02-12)

  • Dictionary literals or dictionary comprehensions in UL4 templates now always create ordered dictionaries (i.e. collections.OrderedDict objects on Python 3.5 and normal dict objects on Python 3.6).

Changes in 5.23 (released 2016-12-16)

  • UL4ON now supports ordered maps.

Changes in 5.22.1 (released 2016-11-02)

  • Fixed the default value for the pysql option --commit.

Changes in 5.22 (released 2016-10-18)

  • PySQL now supports connections to multiple databases via the new pushconnection and popconnection commands (and the connectname key for the PySQL commands procedure, sql, checkerrors, compileall and resetsequence).

  • All PySQL commands now support comments via the "comment" key.

  • The values for the ll.pysql option -v/--verbose have changed: -v1 now is -vdot, -v2 is -vtype and -v3 is -vfull.

Changes in 5.21 (released 2016-09-19)

  • Added a function md5 to UL4.

  • If constant folding doesn’t work for unary or binary operators in UL4, compiling the template no longer fails. Instead the original AST will be used unchanged (and executing the template will then fail).

  • The method ll.color.Color.__add__() has been removed, i.e. color can no longer be added.

  • The method ll.orasql.ForeignKey.pk() has been renamed to refconstraint() and supports foreign keys that reference a unique constraint now.

Changes in 5.20.1 (released 2016-08-04)

  • Fixed a bug in rul4 when database connections are specified on the command line.

Changes in 5.20 (released 2016-07-29)

  • Dictionaries and sets in UL4 now support the clear() method.

  • rul4 now supports saving files to disk, making log calls that print messages to stderr and accessing environment variables. All variables passed to the templates have been moved into an object named globals.

Changes in 5.19.4 (released 2016-06-30)

  • ll.orasql now honors the logging mode of a table or index when creating SQL for it. A new method ll.orasql.Table.logging() has been added to table objects. It returns whether loggging is enabled for this table.

  • ll.orasql.Connection and ll.orasql.connect() now support a new argument decimal. If this argument is true, NUMBERs will be returned as decimal.Decimal objects (otherwise as floats).

Changes in 5.19.3 (released 2016-06-29)

  • Added a new method ll.orasql.Record.replace().

Changes in 5.19.2 (released 2016-06-21)

  • Fixed a bug in ll.orasql.Constraint.names().

Changes in 5.19.1 (released 2016-06-20)

  • The field USER_TAB_COLUMNS.DATA_DEFAULT is different in Oracle 11 and Oracle 12 installations. ll.orasql has been updated to handle the difference.

Changes in 5.19 (released 2016-06-14)

  • The documentation has been ported to Sphinx.

  • ll.pysql now supports var objects with None as the key. Those variables will be used as OUT parameter, but will be thrown away after the procedure call.

Changes in 5.18 (released 2016-05-17)

  • Added a function isexception() to UL4 that returns True if the argument is an exception object.

  • The UL4 exception ll.ul4c.Error has been renamed to LocationError.

  • The __cause__ or __context__ attribute of exception objects now gets exposed to UL4 templates as the cause attribute. In addition to that for the UL4 exception ll.ul4c.LocationError the following attributes get exposed: location (which is the AST node or tag where the error occured), template (the innermost template where the exception occurred), outerpos (the position of the tag where the error occurred) and innerpos (the position of the AST node where the error occurred).

  • The UL4 function type now returns the Python class name for date, color, template exception objects.

Changes in 5.17.1 (released 2016-05-10)

  • Fixed a bug in the query done by orasql.Connection.getobject().

Changes in 5.17 (released 2016-05-04)

  • The internal structure of UL4 templates has changed to simplify exception handling (Text nodes and tags reference the template now instead of only the template source).

  • The rul4 function import has been dropped. Instead two functions load (for reading the content of a file) and compile (for compiling a string into an UL4 template) have been added.

  • A function error for outputting an error message and aborting template rendering have been added to rul4.

Changes in 5.16 (released 2016-04-13)

  • orasql now supports check constraints.

  • orasql now handles inline primary key constraints properly.

  • The scripts oracreate, oradelete, oradrop, oragrant, orareindex and oradiff have a new option --format. The option value pysql switches the output format to PySQL.

  • Procedure and function source code created by ll.orasql will now no longer have a linefeed introduced before the parameter list.

  • orasql.Comment objects now work even if the comment contains linefeeds.

  • orasql.Comment objects now have a method table() that returns the table to which the comment belongs.

  • Some methods in orasql have been renamed: Iterating methods no longer have iter in their name (e.g. itertables() is now simply called tables()). The ddl part of some method names has been changed to sql (e.g. createddl() is now called createsql()).

  • Importing pysql now doesn’t fail if the module pwd or grp doesn’t exist (e.g. on Windows). (However the PySQL file command will still fail if a user/group name is given.)

Changes in 5.15.1 (released 2016-03-21)

  • Fixed some Python 3 compatibility problems in the module ll.daemon and updated the command line argument parsing to make it extensible.

Changes in 5.15 (released 2016-03-18)

  • Calls to UL4 functions and templates now support specifying a * or ** argument multiple times (similar to Python’s PEP 448).

  • Also * and ** expressions are now allowed in list, set and dict “literals”.

  • The UL4 function sorted now supports a key and reverse argument.

  • Strings in UL4 now support the splitlines method.

  • An UL4 function ascii has been added.

  • PySQL no longer supports the -- !!! command terminator. Use the raiseexceptions command instead to specify error handling.

  • The PySQL command setvar now uses the name key as the variable name instead of the var key.

  • A new PySQL command unsetvar has been added for deleting an existing variable.

  • PySQL variables can now be used in expressions, e.g.:

    var('foo_10', 'str').upper()

  • The PySQL function load has been replaced by two functions loadstr for loading strings and loadbytes for loading bytes.

  • orasql.Index now has a method itercolumns() for iterating through the columns of the index.

Changes in 5.14.2 (released 2016-03-02)

  • Fixed color blending in color.Color to use “premultiplied alpha” values. With this change blending colors gives the same result as CSS.

Changes in 5.14.1 (released 2015-12-04)

  • Fixed a bug in ll.make so that Project objects can now be used as arguments in CallAction objects.

Changes in 5.14 (released 2015-12-02)

  • Whitespace handling for UL4 templates has been extended. There are three possible whitespace handling modes now (specified via the new whitespace parameter): "keep" (the old keepws=True) "strip" (the old keepws=False) and the new "smart".

    In smart mode if a line contains only indentation and one tag that doesn’t produce output, the indentation and the linefeed after the tag will be stripped from the text. Furthermore the additional indentation that might be introduced by a for, if, elif, else or def block will be ignored.

    Rendering a template from within another template will reindent the output of the inner template to the indentation of the outer template.

  • Rendering an UL4 template from inside a UL4 template is now again done via the <?render?> tag. To update your code replace <?code r.render()?> with <?render r()?>.

  • Whitespace handling mode for UL4 templates can now be specified in the template source itself via the <?whitepace?> tag:

    <?whitespace smart?>

  • The name and signature of an UL4 template can now be specified in the template source too like this:

    <?ul4 name(x, y, *args, **kwargs)?>

  • Closures in UL4 templates no longer see the state of the variables at the time when the local template was defined, but at the time when it is called. This is similar to most other languages that support closures.

  • In UL4 tags whitespace is allowed now before the tag name, i.e.:

    <? for i in range(10) ?>
   <? print i ?>
<? end for ?>

  • Exposing attributes of objects to UL4 templates can now be customized via the methods ul4getattr() and ul4setattr(). Support for making attributes writable or exposing them under a different name via ul4attrs has been removed.

  • An object can now be made renderable by UL4 templates by implementing the method ul4render().

  • An object can now be made callable by UL4 templates by implementing the method ul4call() (__call__() is still supported).

  • Stacktraces produced by UL4 templates now include less chained exceptions and are much more informative.

  • The rul4 option --keepws has been renamed to --whitespace and defaults to smart now.

  • rul4 got a new option --stacktrace: full displays the full Python stack trace, short (the new default) only displays the exception chain without displaying any Python source.

  • Templates used in rul4 have access to a new function: import, which can be used to load templates from any file.

  • UL4 got two new comparison operators: is and is not for checking for object identity.

  • oradd has been renamed to pysql. The commands are now no longer limited to being on one line. Normal SQL commands are now also supported. Normal SQL commands must be terminated with a comment line starting with -- @@@ and PySQL commands must be either on one line, or start with a line containing only { and end with a line containing only }.

    Three new commands have been added: include includes another pysql file. compileall recompiles all objects in the schema and checkerrors raises an exception if there are objects with compilation errors in the schema.

    Also str/bytes values can be loaded from external files via the load class.

  • If an identifier is given when invoking a sisyphus job it will be included in the log file name now by default.

  • Three new helper functions were added to ll.misc: format_class() formats the name of a class object (e.g. ValueError or http.client.HTTPException). format_exception() formats an exception:

    >>> misc.format_exception(ValueError("bad value"))
'ValueError: bad value'

    exception_chain() traverses the chain of exceptions (via the __cause__ and __context__ attributes).

  • + in the path part of URLs are now considered safe characters. Spaces will be escaped as %20 and no longer as +.

  • ll.orasql.Comment has a new method comment() that returns the text of the column comment itself.

  • The database objects output by ll.orasql.Object.iterreferences() and ll.orasql.Oracle.iterreferencedby() are now sorted by name to get a stable order of dependencies.

  • ll.misc has two new functions: notifystart() and notifyfinish(). The can be used for issuing Mac OS X notifications.

Changes in 5.13.1 (released 2015-06-12)

  • ll.url.URL.relative can now produce “scheme relative” URLs if requested via the parameter allowschemerel, i.e.:

    >>> u1 = url.URL("http://www.example.org/about/index.html")
>>> u2 = url.URL("http://www.example.com/images/logo.png")
>>> u2.relative(u1, allowschemerel=True)
URL('//www.example.com/images/logo.png')

  • The XIST publishing methods now have an additional parameter allowschemerelurls. When allowschemerelurls is true, scheme relative urls are allowed in the output:

    >>> node = html.a(href="http://www.example.org/index.html")
>>> node.bytes(base="http://www.example.com", allowschemerelurls=True)
b'<a href="//www.example.org/index.html"></a>'

Changes in 5.13 (released 2014-12-18)

  • UL4 templates now support signatures. Signatures can be used for top level templates and for subtemplates. This makes it possible to define default values for template variables and to call templates with positional arguments.

  • The option setproctitle for sisyphus jobs has been renamed to proctitle. The new method setproctitle() sets the process title and can be overwritten to customize setting the process title.

  • Locally defined UL4 templates no longer see themselves among the variables of the parent template. This avoids cycles in the object graph.

  • The default for the name parameter in tasks() for sisyphus jobs has changed from str to None, i.e. it defaults to unnamed tasks now.

  • misc.Const now allows to specify a module name.

Changes in 5.12.1 (released 2014-12-09)

  • Fixed a bug in ll.oradd: Printing the final report failed when no commands where executed.

Changes in 5.12 (released 2014-11-07)

  • Fixed bugs in Oracle URLs: the types comment and column are now skipped when iterating a user “directory”. Content in the user directory now works correctly.

  • UL4ON has been reimplemented to be human-readable and more robust against modification of the dumped data. For generating UL4ON dumps with Oracle a PL/SQL package is available at https://github.com/LivingLogic/LivingLogic.Oracle.ul4

Changes in 5.11 (released 2014-10-29)

  • UL4 now supports sets, set literals and set comprehensions.

  • misc.javaexpr() now supports sets.

  • Sisyphus jobs have a new method tasks() that loops over an iterable and calls task() for each item:

    items = sys.modules.items()
for (name, module) in self.tasks(items, "module", lambda kv: kv[0]):
   self.log(f"module is {module}")

  • An option --maxemailerrors has been added to sisyphus jobs: This options limits the number of exceptions and errors messages that will get attached to the failure email.

  • An option --setproctitle has been added to sisyphus jobs: When this options is specified, the process title will be modified during execution of the job, so that the ps command shows what the processes are doing. (This requires that the module setproctitle is installed.)

Changes in 5.10 (released 2014-10-09)

  • Old sisyphus logfiles can now be compressed automatically via gzip, bzip2 or lzma.

  • The functions misc.gzip() and misc.gunzip() have been removed as Python 3.2 has the functions: gzip.compress() and gzip.decompress() which work the same.

Changes in 5.9.1 (released 2014-09-29)

  • Fixed the precedence of the boolean not operator in UL4: Now it has a lower precedence than the comparison operators. i.e. not x in y is parsed as not (x in y).

Changes in 5.9 (released 2014-09-22)

  • A script udiff has been added for doing line by line comparisons of two files or directories. udiff supports all URLs that ll.url supports (e.g. ssh and oracle URLs).

  • The script db2ul4 has been renamed to rul4. The following new features have been added: Additional variables can be passed to the UL4 template via the -D/--define option. Access to Oracle, SQLite and MySQL databases can be disallowed with the options --oracle, --sqlite and --mysql. Executing system commands can be disallowed with the option --system. SQL code that doesn’t return results can be executed with the new Connection method execute(). “out” parameters can now be used via variable objects that can be created with the int(), number(), str(), clob() and date() methods.

  • A new script orareindex has been added that can be used to rebuild/recreate all indexes and unique constraints in an Oracle database.

  • All objects in ll.orasql that represent objects in the database now have a method exists() that returns whether the object exists in the target database.

  • ll.orasql.Index has a new method rebuildddl() that returns SQL for rebuilding the index.

  • URLs have a new walk() method that works similar to the walk() method for XIST trees: walk() is a generator that returns a Cursor object that contains information about the state of the directory traversal and can be used to influence which parts of the directory hierarchy are traversed.

  • The URL methods listdir(), files(), dirs() are generators now.

  • The old URL method walk() has been renamed to walkall() and listdir(), files(), dirs(), walkall(), walkfiles() and walkdirs() have been enhanced:

    listdir(), files() and dirs() now have arguments include and exclude instead of pattern (which worked like include does now). Also patterns can now be lists of strings.

    walkall(), walkfiles() and walkdirs() gained the same arguments. Additionally the arguments enterdir and skipdir can be used to skip directories during traversal.

  • Oracle URLs now support the methods walk(), walkall(), walkfiles() and walkdirs() (with the new arguments include, exclude, enterdir and skipdir). The methods listdir(), files() and dirs() support the arguments include and exclude.

  • The various directory traversal methods in ll.url.URL will now output URLs in sorted order.

  • URL.open() for Oracle URLs now supports the encoding and errors parameter.

  • URLs no longer forward attribute access to unknown attributes to the connection to avoid problems with code that uses hasattr() to check for the presence of an attribute.

  • Fixed handling of the current directory in url.Dir(): url.Dir("") now returns URL('file:./').

  • misc.SysInfo has a new attribute: script_url returns the name of the running script as an ssh URL (e.g. ssh://user@www.example.org/~/project/script.py)

  • The evaluation order of keyword arguments in calls to UL4 functions/templates has been fixed.

  • The test suite for UL4 now runs the Javascript versions of the templates not only on V8 but on Spidermonkey too.

Changes in 5.8.1 (released 2014-06-18)

  • The UL4 function repr now handles recursive lists/dicts similar to Python repr (i.e. it doesn’t raise an exception for infinite recursion).

  • url.URL now handles filenames containing spaces correctly when converting between URLs and filenames.

Changes in 5.8 (released 2014-05-05)

  • UL4 supports while loops now.

  • misc.item() now supports index sequences, which will be applied recursively, so item(["foo", "bar"], (1, -1)) returns 'r'.

  • A new context manager misc.timeout() has been added, that uses signal.alarm() to limit the runtime of the body of the with block.

  • Updated the required version of cssutils to 1.0.

  • Fixed the oraddresetsequence command to really reset the sequence. The parameters minvalue and increment are now optional. If missing, they will be taken from the existing sequence.

  • Passing the clientinfo parameter to cx_Oracle.connect() doesn’t work with Oracle 11.2.0.4.0 (leading to an ORA-03113: end-of-file on communication channel error). The method orasql.connect() has been changed to set the clientinfo parameter after the connection has been established.

  • Fixed cloning of plain XML attributes.

  • Fixed a bug in the C source code that broke compiling with Visual C. From now on we will have Windows installation packages again.

Changes in 5.7.1 (released 2014-02-13)

  • Fixed a bug in the script oradiff that resulting in the wrong order of the output.

  • The oradd command file will now create directories if they don’t exist.

Changes in 5.7 (released 2014-01-30)

  • The ll.oradd command file has been renamed to scp.

  • The new ll.oradd command file will now save the file directly from Python. A file mode, owner and group can be set.

  • The JSON payload of the ll.sisyphus failure email will now be encoded in base64 format to work around a bug in the quoted-printable encoder.

  • To conform to the Python 3 dictionary interface ll.orasql.Record.iterkeys() has been renamed to ll.orasql.Record.keys() and ll.orasql.Record.itervalues() has been renamed to ll.orasql.Record.values(). The original methods ll.orasql.Record.keys() and ll.orasql.Record.values() have been dropped.

Changes in 5.6 (released 2014-01-28)

  • ll.oradd has been updated to support variables and literal SQL in a more direct way. However the old method (via "keys" and "sql") is still supported, but will be removed in one of the next versions.

  • The key "args" is now optional for the oradd commands procedure and sql.

  • Support for oradd dumps in UL4ON format has been removed from ll.oradd.

  • Lines in an oradd dump starting with # will now be ignored.

Changes in 5.5.1 (released 2014-01-27)

  • ll.orasql now unterstands type bodies (so the script oracreate will output them).

Changes in 5.5 (released 2014-01-23)

  • If expressions (i.e. code if cond else code) have been added to UL4.

  • The bitwise operators &, |, ^, ~, << and >> (and their augmented assigment counterparts &=, |=, ^=, <<= and >>=) have been added to UL4.

  • UL4ON now supports slice objects.

  • The oradd script has a new option -d/--directory that is the base directory for file copy actions.

  • oradd now supports executing SQL directly.

  • The project repository is hosted on GitHub now.

Changes in 5.4.1 (released 2013-12-18)

  • Use quoted printable encoding for the JSON attachment in the sisyphus failure email.

Changes in 5.4 (released 2013-11-29)

  • ssh URLs now can handle any version of Python on the remote end. The remotepython parameter has been renamed to python.

  • The default Python version for ssh URLs can now be specified with the environment variable LL_URL_SSH_PYTHON.

Changes in 5.3 (released 2013-10-28)

  • xist.parse.Tidy can now pass the XML declaration and the doctype to the application (however internal DTD subsets will be ignored).

Changes in 5.2.7 (released 2013-10-15)

  • orasql.Record objects are now instances of collections.Mapping and are handled correctly by UL4 now.

Changes in 5.2.6 (released 2013-10-15)

  • Attribute access has been fixed in UL4: For objects that supported the dictionary interface without being a dict, a KeyError was raised before instead of returning an “undefined” object.

Changes in 5.2.5 (released 2013-10-09)

  • starttime and endtime are now included in the JSON data sent with the sisyphus failure report email too.

Changes in 5.2.4 (released 2013-10-09)

  • python_executable and python_version are now included in the JSON data sent with the sisyphus failure report email.

Changes in 5.2.3 (released 2013-10-09)

  • The task() context manager function in sisyphus now allows any UL4 compatible object for the task type and name not just strings or None.

Changes in 5.2.2 (released 2013-10-07)

  • sisyphus now doesn’t reraise the exception if it was handled via email. This means that you will only get one email: either from sisyphus or from your cron daemon. Exceptions that are not instances of Exception will not be handled by sisyphus (i.e. you won’t get an email when you press CTRL-C, but a normal stack trace).

  • In case of a parse error in UL4 templates an exception will now be raised.

Changes in 5.2.1 (released 2013-10-02)

  • Fixed a bug in one of the UL4 templates for sisyphus.

Changes in 5.2 (released 2013-10-01)

  • Added support for bound methods to UL4 templates. This means that methods that should be callable must be included in ul4attrs.

  • UL4 templates now support attribute, item and slice assignment, i.e. the following code will work:

    <?code d = {}?><?code d.foo = 'bar'?>
<?code d = {}?><?code d['foo'] = 'bar'?>
<?code d = [17]?><?code d[0] = 23?>
<?code d = [1, 7, 4]?><?code d[1:2] = [2, 3]?>

  • For objects with attributes exposed to UL4, attributes can be specified as being writable by prepending the name with a + in ul4attrs.

  • Added UL4 functions first and last that return the first or last item produced by an iterable.

  • The default argument for the functions misc.first() and misc.last() now defaults to None. I.e. for empty iterators the default value will always be returned instead of generating an exception.

  • ll.sispyhus can now send an email itself in case of a failure. This email includes information about the failure in plain text, HTML and JSON format.

  • ll.sispyhus now supports subtasks via the method task(). This replaces the prefix() method.

  • ll.sispyhus now creates a relative symbolic link for the current logfile instead of an absolute one.

  • oradd now outputs the keys in its logging output.

  • oradd can now be used to reset sequences.

  • Committing the transactions in oradd can now be done after each record with the new option --commit. --rollback has been removed.

  • Renamed the attributes scriptname and shortscriptname of the misc.sysinfo object to script_name and short_script_name.

  • Fixed the user related attributes of misc.sysinfo.

Changes in 5.1 (released 2013-08-02)

  • The HTML namespace (ll.xist.ns.html) now supports microdata attributes.

  • Added support for triple quoted strings to UL4 templates.

  • Added an UL4 function sum that works like the Python function sum.

  • Variables assigned in the body of a <?for?> loop in UL4 now survive the end of the loop. As a consequence of this, loop variables now leak into the surrounding scope (but not the loop variables for list/dictionary comprehensions or generator expressions).

  • Made checking for recoverable Oracle exceptions in ll.nightshade more robust.

  • Added missing processing instruction class ll.xist.ns.ul4.note.

  • ll.oradd now prints the data object before trying to call the procedure and can handle foreign keys that are NULL.

  • The methods abslum() and rellum() of Color objects are now exposed to UL4 templates.

  • The oradd script has a new option --dry-run to roll back all database changes instead of committing them. This can be used to test whether an oradd dump will work.

  • oradd can now copy files via scp. Parts of the file names used may depend on key values.

  • oradd now supports other out types than integers.

  • The query method for database connections in db2ul4 scripts has changed: Instead of a query and a parameter dictionary, you have to pass in positional arguments that alternate between fragments of the SQL query and parameters. I.e.:

    db.query("select * from table where x=:x and y=:y", x=23, y=42)

    becomes:

    db.query("select * from table where x=", 23, " and y=", 42)

    This makes db2ul4 independent from the parameter format of the database driver.

Changes in 5.0 (released 2013-06-04)

  • The HTML namespace (ll.xist.ns.html) has been updated to support the current HTML5 definition.

    However old elements/attributes from the previous HTML namespace are still supported.

  • XIST now allows arbitrary elements and attributes. ll.xist.parse will parse any XML file, even if the pool object doesn’t contain an element for the element name, and even if an attribute name isn’t declared for an element.

    Undeclared elements will be “plain” instances of ll.xist.xsc.Element (i.e. not instances of a subclass of ll.xist.xsc.Element) with the attributes xmlns and xmlname set accordingly and undeclared attributes will be “plain” instances of ll.xist.xsc.Attr (with proper xmlns and xmlname attributes).

    This new feature requires several API changes which will be described below.

  • Validation is now off by default, to turn it on pass validate=True to parse.tree() or parse.itertree() for parsing, or the publisher object or the bytes(), iterbytes(), string() or iterstring() methods for publishing.

  • Accessing an attribute via __getattr__() (i.e. htmlelement.attrs.class_) only works for attributes that are declared for the class, all other attributes must be accessed via __getitem__() (i.e. htmlelement.attrs["class"]). __getitem__() always requires the XML name of the attribute. __getitem__() also allows an attribute name for a global attribute in Clark notation (i.e. htmlelement.attrs["{http://www.w3.org/XML/1998/namespace}lang"]). A global attribute can also be accessed via a (namespace name, attribute name) tuple (i.e. htmlelement.attrs[("http://www.w3.org/XML/1998/namespace", "lang")]). Using an attribute class or attribute object is also possible (i.e. htmlelement.attrs[xml.Attr.lang] or htmlelement.attrs[xml.Attr.lang('de')]).

  • Using __setattr__() to set attributes only works for declared attributes too. Using __setitem__() to set attributes supports the same kind of arguments as __getitem__() does. For declared attributes the resulting attribute object will always be an instance of the declared attribute class. For all other attributes it will be an instance of ll.xist.xsc.Attr except when an attribute class or instance is used as the key. In this case the attribute will be an instance of that class.

  • The methods convert(), clone(), __copy__(), __deepcopy__(), compacted(), withsep(), reversed(), filtered(), shuffled(), mapped() and normalized() make sure that plain nodes are copied properly, i.e. they retain their custom xmlns and xmlname attributes.

  • The keys in an attribute dictionary (i.e. an ll.xist.xsc.Attrs object) are no longer the attribute classes, but the (namespace name, attribute name) tuples:

    >>> node = html.div({xml.Attrs.lang: 'de'}, id='id42', class_='foo')
>>> list(node.attrs.keys())
[('http://www.w3.org/XML/1998/namespace', 'lang'),
 (None, 'class'),
 (None, 'id')]

  • For all methods that existed in Python/XML pairs (e.g. withnames() and withnames_xml() in xsc.Attrs or elementclass() and elementclass_xml() in xsc.Pool etc.) there is only one version now: A method without the _xml suffix that accepts the XML version of the name.

  • The method checkvalid() has been renamed to validate(). It no longer calls warnings.warn() itself, but is a generator that returns the warning objects. Furthermore the model objects now get passed the complete path instead of only the target node (this is used to implement HTML5’s transparent content model).

  • Validating whether an attribute is allowed is now done in Attrs.validateattr(). The default implementation yields warnings about undeclared local attributes. The HTML5 namespace extends this to also accept any attribute whose name starts with data- or aria-.

  • Node comparison now ignores the classes for elements, entities and processsing instructions, so that plain nodes compare equal to instances of Element, Entity or ProcInst subclasses as long as the name and content of the node matches.

  • ll.xist.parse.Tidy no longer has a skipbad argument.

  • Converter contexts now support string as keys (which must be hierarchical dot-separated names similar to Java package names (e.g. "org.example.project.handler") to avoid name collisions).

  • The docbook module has been updated to support DocBook 5.0.

  • URL objects are pickable now.

  • When whitespace is removed in the literal text of UL4 templates (via the keepws parameter), any initial spaces (before the first line feed) are now no longer removed.

  • If you have Cython installed and the environment variable LL_USE_CYTHON set, several modules will now be compiled into extension modules.

  • It’s now possible to expose attributes and methods of objects to UL4 templates. Exposing attributes can be done by setting a class or instance attribute ul4attrs to a sequence of attribute names. Exposing methods can be done with the decorators ul4c.expose_method() and ul4c.expose_generatormethod().

  • A new UL4 function list has been added. This function works like the Python function list, creating a copy of a sequence or materialzing an iterator.

  • A new UL4 function slice has been added. It works like itertools.slice(), i.e. returning a slice from an iterator.

  • The function html.astext() that converts an XIST tree containing HTML to plain text is now implemented in plain Python so it no longer requires a text mode browser. The function also got more configurable.

  • The objects available to db2ul4 scripts have been changed: oracle, sqlite and mysql are now objects with a connect method that returns a connection object. A connection object now has a method query that executes the query and returns an iterator over the results. Furthermore query supports keyword arguments for parameterized queries, i.e. you can now do:

    <?code db = oracle.connect("user/pwd@db")?>
<?for row in db.query("select * from foo where bar = :bar", bar=42)?>
   <?print row?>
<?end for?>

    The system object now has an execute method that executes the system command.

  • Fixed a bug in orasql.OracleFileResource.close() that surfaced when writing to an Oracle object.

Changes in 4.10 (released 2013-03-04)

  • It’s now possible to use UL4 templates as functions by using the <?return?> tag:

    >>> from ll import ul4c
>>> f = ul4c.Template("<?return 2*x?>")
>>> f(x=42)
84

    It’s also possible to call a template as a function inside another template:

    >>> from ll import ul4c
>>> t = ul4c.Template("<?def x?><?return 42?><?end def?><?print x()?>")
>>> t.renders()
'42'

    Normal output of the template will be ignored if it is used as a function.

    If the template runs through to the end without encountering a <?return?> tag, None will be returned if the template is used as a function.

    If the template is used as a template and a <?return?> tag is encountered executing the template will be stopped (the return value will be ignored).

  • The UL4 tag <?code?> may now contain not only variable assigments, but any other expression. Of course this makes only sense for expressions that have side effects (e.g. a call to the render method).

  • The tag <?render?> has been removed. To update your code replace <?render r()?> with <?code r.render()?>.

  • UL4 functions print and printx have been added. They behave like the respective tags <?print?> and <?printx?>, but can output an arbitrary number of arguments.

  • The builtin UL4 functions are now real objects that can be passed to templates as arguments.

  • The UL4 methods days, seconds, microseconds and months have been added for timedelta/monthdelta objects.

  • Lists in UL4 now support the methods append, insert and pop.

  • Dictionaries in UL4 now support the method update.

  • The db2ul4 script now supports a -w/--keepws argument.

  • The UL4 functions vars and get have been removed.

  • The ** syntax has been removed for UL4 dict literals.

  • The automatic UL4 variable stack has been removed too.

Changes in 4.9.1 (released 2013-01-17)

  • Fixed a bug the printx tag for UL4 templates.

Changes in 4.9 (released 2013-01-17)

  • Fixed a bug in UL4 templates when a template called a top-level template which in turn called its own subtemplate.

  • Fixed and enhanced repr output of UL4 templates and added support for IPythons pretty printing framework.

Changes in 4.8 (released 2013-01-15)

  • Linefeeds and indentation in the literal text for UL4 templates can now be ignored by specifying keepws=False in the template constructor.

Changes in 4.7 (released 2013-01-11)

  • A variable stack is now automatically defined in all UL4 templates. This list contains a stack of the currently executing UL4 templates. stack[-1] is the current template.

  • UL4 templates now support lexical scopes. A locally defined subtemplate can access all local variables of the template in which it is defined.

  • UL4 functions and methods now support keyword arguments, e.g. format(now(), fmt="%Y-%m-%d", lang="en").

  • UL4 templates can no longer be converted to Java CompiledTemplate objects. (However converting it to an InterpretedTemplate is of course still supported).

  • If the view ctx_preferences doesn’t exist orasql.Preference.itername() now will simply return an empty iterator instead of failing with an Oracle exception ORA-00942: table or view does not exist.

  • For sisyphus jobs, the class attribute maxtime can now be set to a datetime.timedelta object.

Changes in 4.6 (released 2012-12-18)

  • The walk() method in XIST has been changed: The return value is a cursor object that provides information about the path and can be used to skip subtrees in the traversal. Filters (which are called selectors now) can no longer influence which parts of the trees are traversed, only whether a node is returned by the iterator or not.

  • itertree() now supports the same interface is the walk() method.

  • A new function filter() has been added that filters the output of walk() or itertree() against a Selector object.

  • The XIST parse events have been renamed: The "start*" events to "enter*" and the "end*" events to "leave*".

  • Slicing XIST elements now returns a sliced element, instead of a slice from the content Frag:

    >>> from ll.xist.ns import html
>>> html.ul(html.li(i) for i in range(5))[1:3].string()
'<ul><li>1</li><li>2</li></ul>'

  • Functions with keyword only arguments are now supported in ll.xist.ns.doc.explain().

  • monthdelta now supports the abs() function (i.e. abs(monthdelta(-1)) returns monthdelta(1).)

Changes in 4.5 (released 2012-11-29)

  • Added UL4 functions any and all.

  • To improve UL4 exception messages there are now several undefined objects, which give information about which key/name/index resulted in the undefined object being created.

  • UL4ON can no longer read or write undefined values.

  • The UL4 function format now swallows all exceptions produced by locale.

  • Oracle URLs now support reading and writing bytes.

  • Because of problems with distribute/pip and pytest ll/__init__.py has been reintroduced.

Changes in 4.4 (released 2012-11-08)

  • Python 3.3 is required now (as the code uses yield from and __qualname__).

  • ll/__init__.py has been removed, i.e. XIST is now a PEP 420 compatible namespace package.

  • Fixed tab/space mix in antlr3/debug.py.

Changes in 4.3.1 (released 2012-11-06)

  • Added a method values to UL4 for dictionaries.

  • Fixed a bug in ll.misc.SysInfo.user_shell.

  • Fixed function ll.xist.ns.doc.explain() for methods.

Changes in 4.3 (released 2012-11-02)

  • UL4 now uses a parser generated by ANTLR instead of using spark. This means that the Python parser can now use the same grammar as the Java parser. (A Python 3 port of the ANTLR runtime is included).

  • Accessing nonexistent variables in UL4 templates now no longer raises an exception but returns the special object Undefined. The same is true for accessing nonexistent dictionary keys or list/string indexes that are out of range.

    In a boolean context Undefined is treated as false and str(Undefined) returns the empty string.

  • Two new UL4 functions have been added: isundefined returns whether the argument is the Undefined object or not. isdefined has the inverted logic, i.e. it returns True if the argument is not the Undefined object.

  • The characters CR and LF are no longer allowed in UL4 string constants. Furthermore the escape sequence \e is no longer supported.

  • All AST nodes for loading constants have been merged into a single class Const.

  • ll.ul4on can now read and write datetime.timedelta and misc.monthdelta objects as well as the new Undefined object from UL4 (ll.ul4c.Undefined).

Changes in 4.2 (released 2012-10-22)

  • UL4 templates now support list and dictionary comprehension as well as generator expressions.

  • A new UL4 function date has been added.

  • The UL4 method join no longer calls str on the items in the argument list.

  • The UL4 function format now supports a third argument: the language for formatting dates. So format(date(2012, 10, 10), '%A', 'de') outputs Mittwoch.

  • UL4 date objects now have a new week method. This method returns the week number of the year. It supports one argument: the weekday number (0 for Monday, … 6 for Sunday) that should be considered the start day of the week. All days in a new year preceding the first week start day are considered to be in week 0. The week start day defaults to 0 (Monday).

  • datetime.timedelta objects are now completely supported in UL4 templates: They can be created with the timedelta function and can be type tested for with istimedelta.

  • Added a new class ll.misc.monthdelta. monthdelta objects can be used to add months/years to a datetime.datetime or datetime.date object. If the resulting day falls out of the range of valid days for the target month, the last day for the target month will be used instead.

  • monthdelta objects are now supported in UL4 templates: They can be created with the monthdelta function and can be type tested for with ismonthdelta.

Changes in 4.1.1 (released 2012-10-04)

  • Fixed a bug in the UL4 handling of slices. (('0' + str(x))[-2:] didn’t work correctly.)

Changes in 4.1 (released 2012-10-02)

  • Loop variable unpacking in UL4 now allows arbitrary nesting.

  • Variable assignment in UL4 now allows variable unpacking too, i.e. (a, b) = [17, 23].

  • The support for Growl notifications in ll.make on the Mac has been replaced by support for Mountain Lions Notification Center via terminal-notifier.

  • sispyhus jobs now support notifications too.

  • Java conversion of ll.ul4c.And has been fixed to evaluate the second operand only when the result isn’t clear from the first.

  • ll.ul4on.Decoder now raises an EOFError when reading from an empty stream.

  • A new script has been added: oradd can be used for importing data into an Oracle database (via procedure calls).

Changes in 4.0 (released 2012-08-08)

  • The source has been ported to Python 3. From now on XIST is a Python 3 only project. A big thanks goes to Martin v. Löwis, who got this conversion started at PyCon DE 2011. He did the basic 2to3 conversion and updated the C source to work on Python 3. Without Martin, XIST wouldn’t have made the leap to Python 3 for several years.

  • As there’s no Python 3 port of libxml2s Python wrapper, XIST now uses lxml for HTML parsing.

    This change shouldn’t have any visible consequences.

  • UL4 templates are no longer compiled to byte code, instead the AST is evaluated or converted to the target sourcecode directly.

  • Generating the final Javascript source code for UL4 templates is now done in Javascript itself.

  • A new module ul4on has been added. This module provides functions for encoding and decoding a lightweight extensible machine-readable text format for serializing the object types supported by UL4.

  • The following new functions have been added to UL4: isfirst, islast, isfirstlast, enumfl. They are variants of enumerate that give information about whether the item is the first and/or last item.

  • The following new functions have been added to UL4: urlquote and urlunquote. They encode/decode the %-escaped form of URL query parameters.

  • The UL4 function json has been renamed to asjson and the following new UL4 functions have been added: fromjson, asul4on and fromul4on.

  • The UL4 function enumerate now supports 1 or 2 arguments (the second argument being the start value).

  • The UL4 functions str, bool, int and float now support being called without arguments (just like in Python).

  • Date constants in UL4 have changed again. They are now written like this: @(2012-04-12) or @(2012-04-12T12:34:56).

  • The <?render?> tag in UL4 now looks like a method call instead of a function call. I.e. <?render t(a=17, b=23)?> has changed to <?render t.render(a=17, b=23)?>.

  • UL4 stacktraces now use exception chaining to report the exception location in nested templates.

  • The UL4 methods find and rfind now support lists and tuples.

  • Two new UL4 functions have been added: min and max.

  • The sort order for attributes when publishing XIST trees can be overwritten by setting the xmlorder class attribute to a string. This string will be used for sorting the attribute. Attributes that have xmlorder set will always be published before those that don’t.

  • Support for the old ipipe infrastructure has been removed. Support for IPythons new pretty printing infrastructure has been added. Output looks like this:

    In [1]: from ll.xist.ns import xml, html
In [2]: html.a(
   ...:         'gurk',
   ...:         xml.Attrs(lang='de'),
   ...:         class_='link',
   ...:         href='http://www.example.org/',
   ...:         id='dings42',
   ...: )
Out[2]:
ll.xist.ns.html.a(
   'gurk',
   ll.xist.ns.xml.Attrs.lang='de',
   class_='link',
   href='http://www.example.org/',
   id='dings42')

  • Added the attributes allowfullscreen and flashvars to ll.xist.ns.html.embed.

  • Added the attribute allowfullscreen to ll.xist.ns.html.iframe.

  • The isdir() method now always returns False for real (i.e. non-file or ssh) URLs. This allows stuff like:

    ucp http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2 \
    ssh://user@www.example.org/~/src/

  • ll.orasql.Index now uses the *_INDEXES views to get a list of all indexes and LOB indexes are filtered out, since they will be recreated with the LOB itself. The method table() has been fixed for indexes that belong to a different user than the index.

  • ll.orasql.LOBStream has a new method seek().

  • ll.make.FileAction supports encoding/decoding when writing/reading the file. For this use the encoding and errors arguments.

  • The XIST node method sorted() has been removed, as it no longer makes sense, because with Python 3 nodes might be uncomparable.

  • Th support for %u escapes in URLs has been removed.

  • The function html.astext() now uses the newer links 2.

  • The scripts oracreate, oradrop, oradelete, oradiff, oramerge, oragrant, orafind and uhpp no longer have an -e/--encoding option. They always use Pythons output encoding.

  • The options -i/--inputencoding and -o/--outputencoding of the script db2ul4 have been replaced with an option -e/--encoding for the encoding of the template files. For printing the result Pythons output encoding is used.

  • The options --inputencoding/--inputerrors and --outputencoding/--outputerrors of ll.sisyphus.Job have been replaced with option --encoding/--errors for the encoding of the log files.

  • oradiff now iterates through the object in correct order, so if you’re running oradiff with -mfull the output shouldn’t produce any errors when executed.

  • ll.orasql.Index can now handle domain indexes.

  • ll.orasql.Preference has been added.

  • ll.orasql now ignores indexes of type IOT - TOP.

  • ll.orasql can now handle primary keys where the underlying index has a different name.

  • ll.orasql now ignores tables with names starting with DR$ (i.e. those created by Oracle Text Search).

  • Attributes of ll.misc.SysInfo instances are now calculated on demand. With this change only one instance of ll.misc.SysInfo is required. This instance is ll.misc.sysinfo.

  • When connecting to the database ll.orasql sets the client_info attribute to the name of the running script (unless clientinfo=None is passed to the connect() call).

  • ll.xist.ns.specials.loremipsum now repeats the text if the specified len attribute is greater that the length of the lorem ipsum text.

Changes in 3.25 (released 2011-08-12)

  • ll.xist.parse.Tidy will now output the attribute events in sorted order. Publishing an XIST node will output the attributes in sorted order too.

  • The compact() method has been renamed to compacted() to avoid collisions with the compact attribute in HTML elements.

  • A new script uhpp has been added, that can be used for pretty printing HTML. As the attributes are output in alphabetical order it can also be used as a tool for comparing HTML files.

Changes in 3.24.1 (released 2011-08-10)

Changes in 3.24 (released 2011-08-09)

  • The ProcInst subclass ll.xist.xsc.AttrProcInst has been replaced with an Element subclass ll.xist.xsc.AttrElement. Conditional handling of the attribute will be used, if the AttrElement instance is the only child of the attribute. Outside of attributes the AttrElement instance will be published normally (via publish(), which must be implemented).

  • ll.xist.ns.ul4.attr_if is an ll.xist.xsc.AttrElement subclass now. The condition is in the cond attribute and the attribute content is inside the element. Outside of an attribute attr_if will put a normal UL4 if condition around its content.

  • ll.xist.ns.ul4.attr_ifnn has been removed.

Changes in 3.23.1 (released 2011-07-28)

Changes in 3.23 (released 2011-07-20)

  • UL4 template objects now have a name. This name will be displayed in exception messages. Nested templates display their own name in the exception message.

  • The module global functions ll.ul4c.compile(), ll.ul4c.load() and ll.ul4c.loads() have been removed. Instead of them the Template constructor and the class methods load() and loads() can be used.

  • The script oradelete now supports the options --include, --exclude and --keepjunk too.

Changes in 3.22 (released 2011-07-14)

  • The scripts oracreate, oradrop and oragrant have new options --include and --exclude that can be used to filter the objects that will be output.

Changes in 3.21 (released 2011-06-03)

  • Oracle 10 doesn’t have a DBA_ARGUMENTS view. Fixed ll.orasql.Function and ll.orasql.Procedure accordingly.

  • The type attribute for the input element now supports the new input types from HTML5.

  • The form elements input, select and textarea gained the additional attributes from HTML5.

Changes in 3.20.2 (released 2011-05-23)

  • Unicode parameters in execute() and executemany() in ll.xist.orasql now get encoded to the Oracle client character set.

Changes in 3.20.1 (released 2011-05-18)

  • Fixed a bug in the Java code generation for UL4 templates: When the template source code contained C-style comments (i.e. /* foo */) invalid Java source code was produced.

Changes in 3.20 (released 2011-05-05)

  • It’s now possible to specify the connection mode (i.e. SYSDBA and SYSOPER) in oracle URLs like this:

    $ uls oracle://sys:pwd:sysdba@dsn/

    Supported are the values normal (the default), sysdba and sysoper.

  • The schema argument used by various methods in ll.orasql has been replaced by a owner argument that can be None (for the current user), the constant ALL for all users (which uses the DBA_* variant of various meta data views if possible or the ALL_* variants otherwise) and a specific user name.

    These views are also used if possible in all spots where the ALL_ views where used before.

  • It’s now possible to list all users in the database with the class methods User.iternames() and User.iterobjects() and with Connection.iterusers().

  • Oracle Column objects have a new method table() that returns the table the column belongs to.

  • Oracle URLs now support the directory user/ which contains all users, i.e. oracle://user:pwd@db/user/ lists all users and oracle://user:pwd@db/user/foo/ lists the same stuff as oracle://foo:pwd@db/. This directory however will not be listed in the root directory oracle://user:pwd@db/.

  • ll.orasql now supports tables without columns.

  • ll.orasql.Table has a new method pk() that returns the primary key constraint (or None if the table has now primary key constraint).

  • A bug in the queries for Index objects in ll.orasql has been fixed.

  • ipipe support has been removed from ll.orasql.

  • Fixed a bug in ll.xist.xsc.Pool: Registered elements/entities etc. now show up as attributes of the pool object.

Changes in 3.19 (released 2011-04-26)

  • ll.orasql now requires cx_Oracle 5.1.

  • If the readlobs option is false for ll.orasql cursors, the CLOBs/BLOBs returned will be wrapped into something that behaves like a Python file.

Changes in 3.18.1 (released 2011-04-13)

  • The methods elements(), procinsts(), entities() and charrefs() of ll.xist.xsc.Pool now handle base pools properly.

Changes in 3.18 (released 2011-04-08)

  • Fixed a regression in ll.orasql.OracleConnection.

  • Fixed ZeroDivisionError in script uls for empty directories.

  • Added a class method ll.orasql.Constraint.iternames() and a class method ll.orasql.Index.iternames() that skips those indexes that are generated by constraints. With this addition uls/ucp now list/copy constraints and indexes properly. All iternames methods now skip objects whose name starts with BIN$.

  • The scripts uls, ucp and ucat have new options --include and --exclude for including/excluding URLs that match a regular expression. They also have an new option --all to include/exclude dot files (i.e. files/directories whose name starts with a dot).

  • ucp now supports to new options --padding and --separator which are used for column output.

  • Two unused options were removed: --verbose from the script ucat and --defaults from the script tld2xsc.

  • ucp -x now prints exception details.

  • The variables available in UL4 templates used by db2ul4 have changed. Instead of a connect object, there are now three objects for each supported database (i.e. oracle, sqlite and mysql)

  • The script doc2txt now reads from stdin and writes to stdout instead of requiring file names on the command line.

  • If the scripts xml2xsc or dtd2xsc are called without arguments stdin is read.

  • ll.xist.ns.rest now handles option lists.

  • The Oracle URLs provided by ll.orasql now have a .sql extension for all schema objects. On writing a .sql extension will be stripped to get the name of the schema object.

  • Oracle URLs now should support schema objects with fancy names (i.e. ones that contain accented characters).

  • ll.orasql.Table has a new method organization() that returns "heap" or normal tables and "index" for index organized tables.

  • Pretty printing of XIST trees can now be customized with the class attributes prettyindentbefore and prettyindentafter. The values will be added to the current indentation level before and after the node in question.

  • All scripts that are part of XIST (uls, ucp, ucat, db2ul4, dtd2xsc, tld2xsc, doc2txt, xml2xsc, oracreate, oradrop, oradelete, oradiff, oramerge, oragrant and orafind) are now properly documented on the webpages.

Changes in 3.17.3 (released 2011-03-02)

  • Enhanced support for table and column names containing non-ASCII characters in ll.orasql.

  • Fixed a bug in the uls script: In long recursive mode files were printed twice.

Changes in 3.17.2 (released 2011-02-25)

  • Fixed setup.py so that the spacer GIF and the UL4 Javascript support library really get installed.

Changes in 3.17.1 (released 2011-02-25)

  • Due to a bug in MANIFEST.in the spacer GIF and the UL4 Javascript support library where not included in the distirbution package. This has been fixed.

Changes in 3.17 (released 2011-02-24)

  • The UL4 function repr now handles all instances of collections.Mapping and collections.Sequence too.

  • The spacer pixel px/spc.gif and the UL4 Javascript support library ul4.js will now be installed alongside the Python modules (in ll.xist.data).

  • The Java source code produced by ll.ul.Template.javasource() will now contain register declarations only for the registers that are actually used.

  • misc.javastring() has been renamed to misc.javaexpr() can can now produce the Java sourcecode for more types.

  • The UL4 method isoformat now omits the time part if it is 00:00:00.

  • The UL4 function repr now produces a valid UL4 date literal for date objects.

  • The UL4 method format is now a function instead.

  • The tests for UL4 now test the Java implementation too.

Changes in 3.16 (released 2011-01-21)

  • The UL4 functions json, type, islist and isdict can now handle all instances of collections.Mapping and collections.Sequence not just tuple, list and dict.

  • ll.sisyphus logging of exceptions and tracebacks should be more robust against encoding problems.

  • The cssutils version has been bumped to 0.9.7.

  • dtd2xsc can now combine the content of more than one DTD into a namespace. Handling of duplicate elements can be specified with a new duplicates option.

  • xml2xsc can now collect the XML info from multiple XML files.

  • Fixed a bug in the command line argument handling of dtd2xsc.

  • dtd2xsc can now handle undefined entities.

  • The help message for all scripts in XIST now show the default for all options.

  • Replaced the function misc.flag() with a class misc.FlagAction that can be used as the action in argparse.ArgumentParser.add_argument() calls.

  • Command line options for all scripts have been enhanced: Flags without a yes/no value now toggle the default (using the new misc.FlagAction).

  • The script xml2xsc has a new option --defaultxmlns for setting a namespace name for elements without a namespace.

  • ll.xist.xnd and the related scripts have seen some refactoring.

Changes in 3.15.3 (released 2010-11-26)

  • ll.sisyphus now supports a non-forking mode (--fork=no). In this mode executing the job and monitoring the maximum runtime is done by the same (single) process.

Changes in 3.15.2 (released 2010-11-25)

  • Publishing an ll.xist.ns.xml.XML object will now always put the correct encoding into the XML declaration, no matter where in the XML tree the xml.XML object sits.

Changes in 3.15.1 (released 2010-11-24)

  • Fixed a bug in the error handling code of the UL4 compiler when an unknown function or method was encountered.

  • Fixed str/unicode problems with the search string in orafind.

Changes in 3.15 (released 2010-11-09)

  • It’s now possible to create Java source code from UL4 templates with the method ll.ul4c.Template.javasource().

  • Creating source code (in Python, Javascript and Java) from UL4 templates has been moved out of ll.ul4c.Template into separate classes.

  • The function ll.xist.ns.fromul4() now uses the new method ll.ul4c.Template.javasource() for generating JSP.

  • The binary format for UL4 templates has changed to enhance readability.

  • ll.xist.ns.jsp.javastring() has been moved to ll.misc.

Changes in 3.14 (released 2010-11-05)

  • UL4 templates now have a method jssource() that returns Javascript source code. This means that now UL4 templates can be converted to: Python source code, JSP source code and Javascript source code.

  • Date constants in UL4 have changed. They are now written like this: @2010-11-05T.

  • ul4c.Template.pythonsource() no longer accepts None as the function name. The output will always be a full function.

Changes in 3.13 (released 2010-10-22)

  • sisyphus jobs now have a new method prefix(). This method is a context manager. For the duration of the with block, the passed in prefix will be prepended to all log lines.

  • ll.sisyphus job can now log to stdout and stderr with the new options -o/--log2stdout and -e/--log2stderr.

  • The tags that ll.sisyphus itself uses for logging have changed slightly. For more info see the module documentation.

  • The option -l for sisyphus jobs has been renamed to -f.

Changes in 3.12.1 (released 2010-10-21)

Changes in 3.12 (released 2010-10-21)

  • The way that ll.sisyphus handles running jobs has changed. Jobs no longer create a pid file. Avoiding duplicate running jobs is done with a file lock on the script file and limiting the maximum runtime is done by forking the process and monitoring the runtime in the parent process. This means that a job that is past its maximum allowed runtime will not be killed by the next job invocation. Instead the job will kill itself.

  • A new class ll.misc.SysInfo has been added that provides host/user/python/script information. ll.sisyphus uses this new class.

  • Changed the default output of tags in ll.sisyphus log files from:

    [tag1, tag2, tag3]

    to:

    [tag1][tag2][tag3]

  • The default location for ll.sisyphus log files has changed to ~/ll.sisyphus/projectname/jobname/.

  • ll.orasql.ForeignKey has a new method itercolumns() for iterating over the columns the foreign key consists of.

  • Fixed a bug in the uls script: For remote URLs uid and gid must be resolved on the remote host.

Changes in 3.11.1 (released 2010-10-18)

Changes in 3.11 (released 2010-10-15)

  • ll.sisyphus has been rewritten. The new version supports: One log file per job invocation; enhanced configuration for logging; command line arguments.

  • Various attributes of UL4 templates are exposed to UL4 itself.

  • Fixed a bug in ll.url.LocalConnection.rename().

Changes in 3.10.1 (released 2010-10-13)

Changes in 3.10 (released 2010-09-24)

  • Python 2.7 is required now as XIST now uses set literals, set and dict comprehension, the new argparse module and various other new features of Python 2.7.

  • All scripts and ll.make have been ported to use argparse.

  • Fixed a bug in ll.nightshade. If the function/procedure didn’t set an encoding, the handling of the response body was totally broken (which resulted in a ISO-8859-1 encoded output).

  • ll.xist.parse.Tidy now supports an additional parameter: If skipbad is true, unknown elements and attributes will be skipped.

  • The random number functions random, randrange and randchoice have been added to UL4.

  • A new function ll.misc.prettycsv() has been added. It can be used to pretty print the data produced by the csv module.

Changes in 3.9 (released 2010-08-04)

  • ll.xist.ns.html.html will no longer change the lang and xml:lang attributes. This functionality has been moved to the new element ll.xist.ns.htmlspecials.html. Furthermore this new element won’t change existing attributes.

  • ll.xist.ns.html.title no longer does any manipulation of its content.

  • The Java string literal formatting function in ll.xist.ns.jsp has been exposed as javastring().

  • Fixed a bug in oracreate: If the source of procedures and functions didn’t have whitespace between the name and the ( the ( was missing from the output.

Changes in 3.8.3 (released 2010-07-29)

  • str arguments are now always treated as BLOBs in ll.orasql functions and procedures.

Changes in 3.8.2 (released 2010-06-21)

  • Fixed a bug in the logging methods of ll.sisyphus.Job: Logging unicode strings didn’t work. Now all strings are promoted to unicode.

  • The default encoding for ll.sisyphus log files has changed to UTF-8. This can be changed by setting the class attribute encoding in the class derived from ll.sisyphus.Job.

Changes in 3.8.1 (released 2010-06-17)

  • The method ll.url.URL.import_() that had been dropped in version 3.8 has been reintroduced. However internally misc.module() is used for creating the module object. A side effect of this is that importing from non-local URLs now works:

    >>> from ll import url
>>> u = url.URL("http://www.livinglogic.de/Python/misc/index_module.py")
>>> m = u.import_()
>>> m.last("gurk")
"k"

Changes in 3.8 (released 2010-06-15)

  • The parsing infrastructure has been completely rewritten to be more modular and to support iterative parsing (similar to ElementTree).

    Now parsing XML is done in a pipelined approach that looks like this:

    >>> from ll.xist import xsc, parse
>>> from ll.xist.ns import html
>>> doc = parse.tree(
...   parse.String("<a href='http://www.python.org/'>Python</a>")
...   parse.Expat()
...   parse.NS(html)
...   parse.Node(pool=xsc.Pool(html))
... )
>>> doc.bytes()
'<a href="http://www.python.org/">Python</a>'

    Iterative parsing looks like this:

    >>> from ll.xist import xsc, parse
>>> from ll.xist.ns import xml, html, chars
>>> for (evtype, path) in parse.itertree(
...   parse.URL("http://www.python.org/"),
...   parse.Expat(ns=True),
...   parse.Node(pool=xsc.Pool(xml, html, chars)),
...   filter=html.a/html.img
... ):
...   print path[-1].attrs.src, "-->", path[-2].attrs.href
http://www.python.org/images/python-logo.gif --> http://www.python.org/
http://www.python.org/images/trans.gif --> http://www.python.org/#left%2Dhand%2Dnavigation
http://www.python.org/images/trans.gif --> http://www.python.org/#content%2Dbody
http://www.python.org/images/donate.png --> http://www.python.org/psf/donations/
http://www.python.org/images/worldmap.jpg --> http://wiki.python.org/moin/Languages
http://www.python.org/images/success/tribon.jpg --> http://www.python.org/about/success/tribon/

  • The XIST element ll.xist.ns.specials.z has been moved to the ll.xist.ns.doc module.

  • The function ll.xist.xsc.docprefixes has been dropped. A new function ll.xist.xsc.docpool has been added.

  • The module ll.xist.parsers has been renamed to parse.

  • The module ll.xist.presenters has been renamed to present.

  • The classes ll.xist.converters.Converter and ll.xist.publishers.Publisher has been moved to ll.xist.xsc. The modules ll.xist.converters and ll.xist.publishers no longer exist.

  • The walk methods walknode() and walkpath() have been renamed to walknodes() and walkpaths() and the implemention has been moved from the nodes classes into WalkFilter. WalkFilter has been moved to ll.xist.xfind.

  • A new selector has been added to ll.xist.xfind: AnySelector outputs all nodes.

  • Added a new function misc.module() that creates a module from source code.

  • ll.url.Path has been simplified: Path segments are strings instead of tuples now.

  • The old URL method import_() has been removed. The new function misc.module() can now be used for that.

  • The two classes ll.make.PoolAction and ll.make.XISTPoolAction have been dropped. You can use make.ObjectAction(misc.Pool).call() and make.ObjectAction(xsc.Pool).call() for that.

  • The class XISTParseAction has been removed. This action can be replaced by a combination of ObjectAction, CallAction and CallAttrAction.

  • Two new UL4 functions abs and utcnow have been added.

  • A few methods have been added to UL4 date objects: mimeformat, day, month, year, hour, minute, second, microsecond, weekday and yearday.

  • Use autoboxing in the Java code generated by ll.xist.ns.jsp.fromul4.

  • All code has been switched to using the format() method instead of using the % operator.

  • ssh URLs in ll.url now use the standalone execnet package.

  • ssh URLs now support a nice argument instead of ssh_config.

Changes in 3.7.6 (released 2010-05-14)

Changes in 3.7.5 (released 2010-04-19)

  • ll.orasql.PrimaryKey has a new method itercolumns() that returns an iterator over the columns this primary key consists of.

Changes in 3.7.4 (released 2010-03-25)

Changes in 3.7.3 (released 2010-02-27)

Changes in 3.7.2 (released 2010-02-26)

  • Fixed two bugs in the XML codecs:

    • An externally specified encoding wasn’t honored in the incremental decoder.

    • Fixed reset() for incremental codecs: If encoding has been changed during parsing in the incremental codecs it now gets reset to its proper initial value.

  • Fixed a bug in the handling of the UL4 opcode addlist in ll.xist.ns.jsp.fromul4().

  • Added missing processing instruction class for the UL4 def tag to the ll.xist.ns.ul4 namespace module.

  • The generated JSP code for the loadvar opcode now uses the Java method Utils.getItem, so that non-existent variables no longer get treated as None/null.

Changes in 3.7.1 (released 2010-02-08)

  • ll.xist.ns.jsp.fromul4() now outputs the correct code for calling the format method on date objects (This requires version exp-22 of the UL4 Java package).

Changes in 3.7 (released 2009-09-10)

  • In UL4 templates it’s now possible to define locale templates via <?def tmpl?>templatecode<?end def?>.

  • Python 2.6 is required now.

  • ll.orasql and ll.nightshade are now part of the distribution.

  • ll.make has a new Action class: ObjectAction simply returns an existing object.

  • The following classes have been removed from ll.make: EncodeAction, DecodeAction, EvalAction, GZipAction, GUnzipAction, JavascriptMinifyAction, XISTBytesAction, XISTStringAction, JoinAction, UnpickleAction, PickleAction, TOXICAction, TOXICPrettifyAction, SplatAction, UL4CompileAction, UL4RenderAction, UL4DumpAction, UL4LoadAction, XISTTextAction and XISTConvertAction. All of these actions can be executed by using CallAction or CallAttrAction.

  • ll.make.PipeAction has been renamed to TransformAction.

  • The new ll.make.PipeAction pipes the input through an external command.

  • ll.make.FileAction now automatically wraps the key argument into an URL object.

  • ll.make.FileAction has two new methods chmod() and chown() that return a ModeAction and OwnerAction for modifying the file created by the FileAction.

  • ll.make.Action has three new methods: call(), getattr() and callattr() create a CallAction, GetAttrAction or CallAttrAction object respectively.

  • The division operator is no longer implemented for Action objects in ll.make.

  • Two new UL4 functions have been added: float and iscolor.

  • Two new scripts have been added: uls can be used to list any directory given as an URL. ucat can be used to output any file or directory.

  • The script ucp now changes the user and group only if a user or group is given.

  • A bug in the 64-bit support for sgmlop has been fixed.

  • Fixed a bug in the remote stat() method for ssh URLs (it seems that the posix.stat_result tuple objects can no longer be pickled).

  • There’s a new function misc.itersplitat() for splitting a string at specified positions.

  • For ssh URLs a keyword argument ssh_config is supported now instead of identity (This mirrors the corresponding change in the py library)

Changes in 3.6.6 (released 2009-07-09)

  • Fixed handling of empty pid files in ll.sisyphus (Fixes issue #11 reported by Jarek Zgoda).

Changes in 3.6.5 (released 2009-06-02)

  • Fix UL4 templates that produce no output: As the generated Python sourcecode didn’t contain any yield statements, the resulting function was an ordinary function instead of a generator.

Changes in 3.6.4 (released 2009-03-19)

  • A new UL4 method join has been added. It works like the Python string method join.

  • ll.misc has three new functions: gzip() und gunzip() can be used for compressing and uncompressing byte strings with gzip. jsmin() can be used to minify Javascript source.

  • Parsing an empty string with tidy=True in ll.xist.parsers.parsestring() now works again.

Changes in 3.6.3 (released 2009-03-02)

  • The xfind operators attrhasvalue, attrhasvalue_xml, attrcontains, attrcontains_xml, attrstartswith, attrstartswith_xml, attrendswith, attrendswith_xml, hasid and hasclass now support multiple values. The operator matches the node if it matches with any of the given values.

  • A new function reversed is now available in UL4 templates. It returns an iterator that will output the items of any sequence in reverse order.

Changes in 3.6.2 (released 2009-02-16)

  • Inside UL4 templates rendering other templates can now be done with the new render method. This method returns the template output as a string. Passing parameters can be done via keyword arguments or with the ** syntax like when using the render tag.

  • A new version of the int function has been added to UL4: When called with two arguments, the first must be a string, and the second is treated as the base for the conversion.

Changes in 3.6.1 (released 2009-01-27)

  • Generating the Python source from an UL4 template is now 20-25% faster.

  • Fixed a buffer overrun in the C portions of the url module.

  • Added a class addattr to ll.xist.xsc. This can be used to extend XML attributes via with blocks.

  • Added the function ll.xist.ns.jsp.fromul4() which can turn an UL4 template into JSP source code.

Changes in 3.6 (released 2008-12-31)

  • The following Color class methods have been dropped: fromrgba, fromrgba4, fromrgba8, fromint4, fromint8.

  • The following Color properties have been dropped: r4, g4, b4, a4, r8, g8, b8, a8, r, g, b, a int4, int8, rgb4, rgba4, rgb8, and rgba8. The new methods r, g, b and a return the 8 bit component values.

  • The class methods fromhsva and fromhlsa have been renamed to fromhsv and fromhls.

  • The property css has been dropped. Instead the CSS string is returned by __str__.

  • Dividing colors now does a scalar division. Blending colors is now done with the modulo operator.

  • Support for color objects has been added to UL4.

  • The XPIT templating language and ll.make.XPITAction have been removed.

  • Fixed a bug in ll.make.CacheAction.get(): The action must return real data when called with bigbang as the timestamp.

  • ll.make.UL4RenderAction has been fixed.

Changes in 3.5 (released 2008-12-05)

  • A new function json has been added to UL4: This function returns a JSON dump of the object passed in (this requires either simplejson or Python 2.6).

  • The UL4 function csvescape has been renamed to csv.

  • A new option --showregistration/-r has been added to make scripts.

  • ll.make now supports Growl notifications on Mac OS X. To activate it set the LL_MAKE_GROWL environment variable to 1 or use the -g or --growl options.

  • ll.make has a new action class JavascriptMinifyAction for minimizing Javascript source.

  • ll.color.Color has been rewritten to create immutable objects with the components being 8 bit values (i.e. 0-255) instead of floating point values between 0 and 1. An alpha component has been added.

  • A strong element has been added to the ll.xist.ns.doc namespace.

Changes in 3.4.4 (released 2008-09-16)

  • Fixed a bug in ll.make.JoinAction.execute().

Changes in 3.4.3 (released 2008-09-09)

  • css.applystylesheets() could no longer handle style declarations containing comments. This has been fixed now.

Changes in 3.4.2 (released 2008-09-03)

  • Parsing didn’t work when tidy was set to true and a base argument was given. This has been fixed now.

Changes in 3.4.1 (released 2008-08-29)

  • Bugs with thread local storage have been fixed so using xsc.Pool, xsc.build and URL contexts in with blocks in multithreaded applications should work now.

Changes in 3.4 (released 2008-08-19)

  • Templates can no longer be passed as a separate dictionary to UL4 templates but are passed as variables like other data objects too.

  • Strings in UL4 have gained a new method capitalize.

  • Printing XML escaped strings in UL4 has now gained its own tag and opcode. <?printx foo?> is equivalent to <?print xmlescape(foo)?>.

  • Exception handling in UL4 has been rewritten to allow proper error reporting when calling nested templates.

  • UL4 has gained a new function zip. It can be called with two or three arguments and does what itertools.zip() does.

  • UL4 has gained another new function: type returns the type of its argument as a string.

  • UL4 now supports tuple unpacking in <?for?> tags with three variables.

  • UL4 has a new tag for comments: <?note This is comment?>.

  • A new script db2ul4.py has been added that can render UL4 templates with database content.

  • In UL4s <?render?> tags it’s now possible to pass along a complete argument dictionary via the **arg syntax just like in Python. This syntax can even be used multiple times in the call. This syntax is available in dictionary literals too, i.e. {1:2, 3:4} and {**{1:2}, **{3:4}} are equivalent.

  • A new UL4 function get has been added that works similar to the dictionary method get, but works with global variables.

  • The missing processing instruction render has been added to ll.xist.ns.ul4.

  • xml_codec now partially works, even if the C module is missing. As long as you explicitly specify an encoding on parsing and publishing it should work.

  • A new processing instruction class ll.xist.AttrProcInst has been introduced. When an AttrProcInst node is the first node in an attribute, it takes over publishing of the attribute. In all other cases the processing instruction disappears completely. UL4 uses this to implement “conditional attributes” (via the new classes attr_if and attr_ifnn).

  • Building trees with with blocks has changed slightly. Nodes used in with blocks and with + are now passed to a with handler instead of building the tree directly. This fixes a problem when nested convert() calls use with blocks.

  • The element ll.xist.ns.form.memo has been renamed to textarea and ll.xist.ns.form.edit has been renamed to text. Classes ll.xist.ns.form.button and ll.xist.ns.form.file have been added.

  • Iterating through the inputs in ll.make actions has been fixed (i.e. the additional inputs will be output too). ll.make.Project.findpaths() has been fixed to work with non-ll.make.Action inputs. (This means that now you have to pass a real registered target action to findpaths() not just its key).

  • ll.make has gained a new action: XISTStringAction publishes an XIST node as a unicode string. XISTPublishAction has been renamed to XISTBytesAction.

  • Fixed a bug in the caching logic in ll.make.CacheAction().

  • ll.make.CallMethAction has been renamed to CallAttrAction because it can be used to e.g. call functions in a module too.

  • The properties showaction, showstep and shownote of ll.make.Project object can now be assigned booleans as well (which results in all or no actions being shown.

  • The version number for cssutils has been bumped to 0.9.5.1.

Changes in 3.3.2 (released 2008-07-15)

  • Dictionaries in UL4 have gained a new method get.

  • The version number for cssutils has been bumped again (to 0.9.5rc2 or a later 0.9.5 version).

  • Fixed a bug in the parsing of slice expressions in UL4.

  • ll.make has gained a new UL4RenderAction action.

  • Fixed a bug in the formatting for the getslice2 opcode for UL4.

Changes in 3.3.1 (released 2008-07-14)

  • Fixed a bug in the implementation of the “not” operator in UL4.

  • When the UL4 compiler encounters unclosed blocks, it will now include the start location of the block in the error message.

Changes in 3.3 (released 2008-07-11)

  • XIST has gained its fourth templating language: UL4 the “Universal Layout Language”. This templating language is similar in capabilities to Djangos templating language. However UL4 templates are compiled to a bytecode format, which makes it possible to implement template renderers in other languages and makes the template code “secure” (i.e.template code can’t open or delete files).

  • ll.make has gained new actions: GZipAction, GUnzipAction, CallFuncAction, CallMethAction, UL4CompileAction, UL4DumpAction and UL4LoadAction.

  • The version number for cssutils has been bumped to 0.9.5rc1.

  • Nodes of type ll.xist.xsc.Comment and ll.xist.xsc.DocType inside of attributes are now simply ignored when publishing instead of generating an exception.

  • All actions in ll.make no longer check whether their inputs are action objects. Non-action objects are simply treated as ancient input data. This also means that most action classes have an input parameter in their constructor again, as this input could now be a constant.

  • Most attributes of action objects in ll.make can now be action objects themselves, so for example the name of the encoding to be used in an EncodeAction can be the output of another action.

  • ll.make.ImportAction has been dropped as now the module object can be used directly (e.g. as the input for an XISTPoolAction object).

  • ll.misc.xmlescape() now escapes ' as &#39; for IE compatibility.

  • Functions ll.misc.xmlescape_text() and ll.misc.xmlescape_attr() have been added that implement the functionality from XIST 3.2.5 and earlier.

  • The default parser for XIST is expat now. To switch back to sgmlop simply pass an SGMLOPParser object to the parsing functions:

    >>> from ll.xist import parsers
>>> node = parsers.parsestring("<a>", parser=parsers.SGMLOPParser())

  • TOXIC has been split into a compiler module ll.toxicc and an XIST namespace ll.xist.ns.toxic. TOXIC now supports output for SQL Server. The function xml2ora() as been renamed to compile() (and has a new mode argument for specifying the database type).

  • The targetroot parameter for ll.make.XISTConvertAction.__init__() has been renamed to root.

Changes in 3.2.7 (released 2008-05-16)

  • Added the missing file _misc_include.c to the distribution archives.

Changes in 3.2.6 (released 2008-05-07)

  • A new action class EvalAction has been added to ll.make.

  • ll.xist.helpers.escapetext() and ll.xist.helpers.escapeattr() have been merged into one function that escapes all special characters (including ' and ") and has been renamed/moved to ll.misc.xmlescape().

  • Python versions of all the functions in the module ll.misc have been added. Those versions will be used in case the C module is not available.

Changes in 3.2.5 (released 2008-04-11)

  • A refcounting bug in the attribute parsing code of sgmlop has been fixed.

  • The helper function cssescapereplace() has been removed, as it’s no longer needed.

  • Pure Python versions of helpers.excapetext() and helpers.escapeattr() have been added, in case the C module is not available.

Changes in 3.2.4 (released 2008-04-02)

  • The following functions have been added to ll.xist.css: parsestring(), parsestream(), parsefile(), parseurl() and write(). They parse CSS resources and are similar to the XML/HTML parsing functions in that they apply the specified base URL to all URLs in the style sheet.

  • cssutils 0.9.5b2 is required now.

  • ll.xist.css.iterrules() and ll.xist.css.applystylesheets() now support specifying whether the preferred stylesheets or an alternate stylesheet group should be used.

  • ll.xist.xsc.ProcInst.__mul__() and ll.xist.xsc.ProcInst.__rmul__() now return a fragment containing the node repeated a number of times instead of one processing instruction node containing repeated content.

  • The constructor for ll.xist.parsers.ExpatParser now takes two additional arguments:

    xmldecl

    If this is true the XML declaration will appear in the resulting XIST tree.

    doctype

    If this is true the doctype declaration will appear in the resulting XIST tree (however any internal DTD subset will be dropped).

Changes in 3.2.3 (released 2008-03-04)

  • cssutils 0.9.5 is used now. This simplifies the implementation of css.selector().

  • A function ll.xist.css.geturls() has been added. This returns a list of all the URLs in a cssutils stylesheet.

  • toxic.xml2ora() now treats unknown processing instructions as text. This makes it possible to e.g. output an XML header via toxic.

  • The pseudo-elements in ll.xist.ns.jsp are no longer in a namespace, so they will always be published without any prefixes.

Changes in 3.2.2 (released 2008-02-25)

  • A new method replaceurls() has been added to ll.xist.xsc.StyleAttr. With this method all URLs in a style attribute can be replaced.

  • Fixed a bug in ll.xist.parsers.SGMLOPParser.begin(): The encoding wasn’t passed properly to the XML decoder.

  • ll.xist.xsc.ProcInst.publish() now calls the checkvalid() method too.

Changes in 3.2.1 (released 2008-02-05)

  • It’s now possible to force the publisher to output certain xmlns attributes via the showxmlns argument to the Publisher constructor.

Changes in 3.2 (released 2008-02-01)

  • The core package has been moved into XIST, installing XIST now only requires one package.

  • ll.toxic has been moved into XIST and is now available as ll.xist.ns.toxic.

  • When a ll.make.XISTParseAction object is executed the content of the pool will now be extended by the content of the pool from the XISTPoolAction instead of being replaced.

  • ll.make.Pool and ll.xist.xsc.Pool no longer use a WeakValueDictionary, but a simple dict. This means they can now store any object. A method clear() has been added, which removes all registered objects.

  • Fixed a bug in ll.xist.css.iterrules() that surfaced when a base argument was given.

  • Fixed a second bug in ll.xist.css.iterrules() where the href of a link element wasn’t applied to the URLs in the stylesheet.

Changes in 3.1 (released 2008-01-18)

  • Fixed the problem that the source distibution didn’t include header files.

  • If an URLAttr attribute contains a processing instruction XIST will no longer transform the URL in any way.

  • Fixed a parser bug where attributes were dropped when the attribute value was empty.

  • Putting a module into a Pool object now copies the xmlns attribute too. This makes it possible to use Pool objects as conversion targets.

Changes in 3.0 (released 2008-01-07)

  • Namespaces have been greatly simplified. There are no namespace modules any longer. An element class can be assigned a namespace by setting the xmlns class attribute to the namespace name. Global attributes can be assigned a namespace by setting the xmlns attribute on the attribute class itself (not on the Attrs class). The classes Prefixes and NSPool are gone too. Instead a new class Pool is used to specify which classes should be used for parsing.

  • Dependency on PyXML has finally been dropped. XIST now uses its own XML parsing API. Two parsers are available: One based on expat and one based on a custom version of sgmlop.

  • Tree traversal has been rewritten again. XFind expressions involving multiple uses of // now work correctly. The method walk() now doesn’t yield Cursor objects, but simple path lists (actually it’s always the same list, if you want distinct lists use walkpath()). Applying XFind expressions to nodes directly is no longer supported, you have to call walk(), walknode() or walkpath() with the XFind expression instead. Many XFind operators have been renamed and/or reimplemented (see the documentation for the xfind module for more information).

  • The methods __getitem__(), __setitem__() and __delitem__() for Frag and Element now support the new walk filters, so you can do:

    • del node[html.p] to delete all html.p child elements of node;

    • del node[html.p[2]] to delete only the third html.p;

    • node[xfind.hasclass("note")] = html.p("There was a note here!") to replace several child nodes with a new one;

    • for c in node[xfind.empty]: print c.bytes() to print all empty (element) children of node;

    • del node[node[0]] to delete the first child node (which is silly, but illustrates that you can pass a node to get/replace/delete that node);

  • A new module ll.xist.css has been added which contains CSS related functionality: The generator function iterrules() can be passed an XIST tree and it will produce all CSS rules defined in any html.link or html.style elements or imported by them (via the CSS rule @import). This requires the cssutils package.

  • The function applystylesheets() modifies the XIST tree passed in by removing all CSS (from html.link and html.style elements and their @imported stylesheets) and putting the styles into style attributes of the affected elements instead.

  • The function selector() returns a tree walk filter from a CSS selector passed in as a string.

  • Constructing trees can now be done with with blocks. Code looks like this:

    with xsc.Frag() as node:
   +xml.XML()
   +html.DocTypeXHTML10transitional()
   with html.html():
      with html.head():
         +meta.contenttype()
         +html.title("Example page")
      with html.body():
         +html.h1("Welcome to the example page")
         with html.p():
            +xsc.Text("This example page has a link to the ")
            +html.a("Python home page", href="http://www.python.org/")
            +xsc.Text(".")

print node.conv().bytes(encoding="us-ascii")

    Also the function xsc.append() has been renamed to add() and supports with blocks now instead of XPython.

  • A subset of ReST is supported now for docstrings when using the ll.xist.ns.doc module. The module attribute __docformat__ is now honored (Set it to "xist" to get XIST docstrings).

  • Many classes in the ll.xist.ns.doc have been renamed to more familiar names (from HTML, XHTML 2 or ReST).

  • The media attribute of html.link and html.style now has a method hasmedia().

  • The node method asBytes() has been renamed to bytes() and bytes() has been renamed to iterbytes().

  • The node method asString() has been renamed to string() and a new method iterstring() has been added.

  • ll.xist.ns.xml.XML10 is gone now. Use ll.xist.ns.xml.XML instead.

  • xsc.tonode() now will raise an exception when it can’t handle an argument instead of issuing a warning.

  • A class attribute empty inside element classes will now no longer get converted into model.

  • ll.xist.ns.doc.pyref now copes better with decorated methods.

  • The deprecated Element methods hasAttr(), hasattr(), isallowedattr(), getAttr(), getattr(), setDefaultAttr(), setdefaultattr(), attrkeys(), attrvalues(), attritems(), iterattrkeys(), iterattrvalues(), iterattritems(), allowedattrkeys(), allowedattrvalues(), allowedattritems(), iterallowedattrkeys(), iterallowedattrvalues(), iterallowedattritems() and copyDefaultAttrs() have been removed. The deprecated Attrs method copydefaults() has been removed too.

  • The namespace module ll.xist.ns.cond has been removed.

  • When calling the function ll.xist.parsers.parseURL() the arguments headers and data are now passed along to the parser’s method only if they are specified. This makes it possible to pass ssh URLs to ll.xist.parsers.parseURL().

  • The methods withnames() and withoutnames() have been split into two that take Python names and two that take XML names. Multiple arguments are used now (instead of one argument that must be a sequence). Passing a namespace to remove all attributes from the namespace is no longer supported.

  • The Attrs methods updatenew() and updatexisting() have been removed.

Changes in 2.15.5 (released 2007-07-17)

  • The Python quotes example no longer contains the XML source or the generated HTML.

Changes in 2.15.4 (released 2007-07-16)

  • The Python quotes example now always parses the file from the original URL.

  • The Python quotes and the media example now print the result to stdout.

Changes in 2.15.3 (released 2007-07-16)

  • Use a consistent license (MIT) everywhere. This should make XIST Debian compatible.

  • Change the Python quotes example, so that it works even if there’s no python-quotes.xml in the current directory.

Changes in 2.15.2 (released 2007-01-24)

  • Fixed a bug in presenters.CodePresenter.__str__().

  • Fixed base URL handling for tidy parsing.

  • Updated examples.

  • Updated xiter() and xattrs() implementations for Node and Namespace to conform to the newest version of IPython.

Changes in 2.15.1 (released 2006-09-25)

  • Fixed a few bugs in the sgmlop function declarations.

  • Readded the spacer pixel.

Changes in 2.15 (released 2006-09-24)

  • XIST has been made compatible with Python 2.5: Code has been updated to use the proper C API for memory management and PEP 353 support has been added. XIST now includes its own fixed version of sgmlop.

  • The ll.xist.xsc.Attrs methods with() and without() have been renamed to withnames() and withoutnames() for Python 2.5 compatibility.

  • ll.xist.ns.htmlspecials.pixel no longer handles colors via different GIFs. It uses the background-color in the style attribute instead. The same change has been implemented for ll.xist.ns.htmlspecials.autopixel. It’s now possible to overwrite the default src attribute value of root:px/spc.gif either via the XML attribute or via the converter context.

  • The node method asText() has been made a function, moved into the html namespace and renamed to astext(). Furthermore elinks is used for plain text formatting now instead of w3m.

Changes in 2.14.2 (released 2006-07-04)

  • Fixed a bug in the presentAttr() method of ll.xist.presenters.TreePresenter.

Changes in 2.14.1 (released 2006-06-29)

  • Fixed a bug in the presentEntity() method of ll.xist.presenters.CodePresenter.

  • Updated installation instructions.

Changes in 2.14 (released 2006-06-28)

  • Namespaces for RSS 0.91, RSS 2.0 and Atom 1.0 have been added.

  • A new namespace ll.xist.ns.detox has been added that is similar to ll.toxic but can be used to generate Python code instead of PL/SQL code. Using detox templates is about 50 times faster than using XIST trees directly and about 10 times faster than Kid.

  • Presenters are now compatible to IPython ipipe module. This means that you can browse XIST trees interactively if you have IPython installed. NormalPresenter and the Node methods repr() and asrepr() have been removed.

  • A new processing instruction ll.xist.ns.specials.url has been added that does the same URL transformation as ll.xist.xsc.URLAttr does.

  • On publishing ll.xist.ns.html.html now only adds a lang and xml:lang attribute, if neither of them exists.

  • setuptools is now supported for installation.

Changes in 2.13 (released 2005-10-31)

  • ll.xist.xsc.Namespace.tokenize() requires a unicode object as input now. This makes it possible to use encodings that are not ASCII compatible (such as UTF-16). The encoding argument is gone.

  • ll.xist.xsc.Node.asString() uses the encoding argument to determine which characters have to be output as character references now. (You’ll still get a unicode object as the result.)

  • A new processing instruction class ll.xist.ns.specials.literal has been added, that will output its content literally when published. This can be used for embedding preformatted XML (e.g. from a database) into an XIST tree.

Changes in 2.12 (released 2005-10-13)

  • Namespaces for Relax NG and Kid have been added.

  • XIST requires version 1.0 of the core package now.

  • The class name for the DocBook DTD class has been fixed.

Changes in 2.11 (released 2005-07-29)

  • A script xml2xsc.py has been added, that can be used to parse an XML file and generate a rudimentary XIST namespace from it.

  • A DocType for XHTML 1.1 has been added (suggested by Elvelind Grandin).

  • Line number information is now added when parsing HTML.

  • The sorted() method now supports the same arguments (cmp, key and reverse) as list.sort() and sorted() in Python 2.4.

  • The walk() doesn’t yield the node directly, but yields a Cursor object now, with has several ways of referencing the node.

  • New methods walknode(), walkpath() and walkindex() have been added.

  • Presenters use an iterator API instead of a stream API now. Dumping an XML tree presentation to the terminal can now start immediately instead of having to wait for the complete string to be formatted.

  • Fixed a bug with element/attribute names that contained a . character. (This broke ll.xist.ns.fo.)

  • Fixed a bug with xmlns attributes in nested elements. When an element ended the parser restored the wrong prefix mapping.

  • The python-quotes demo has been updated to use the current version of AMK’s XML file.

  • Removed iterator stuff from ll.xist.xfind, as this is now part of the ll package/module.

  • The function ToNode() has been renamed to tonode().

  • ll.xist.Context no longer subclasses list.

  • ll.xist.ns.doc.explain will now try to output the objects in the order in which they appear in the Python source.

  • The node methods find() and findfirst() have been removed.

  • ll.xist.ns.cond now uses a sandbox dictionary in a converter context for evaluating expression.

Changes in 2.10 (released 2005-05-20)

  • The content of the processing instruction ll.xist.ns.code.pyexec will not be executed at construction time, but at conversion time. The code in ll.xist.ns.code.pyexec or ll.xist.ns.code.pyeval will no longer be executed in the ll.xist.sandbox module (which has been removed), but in a sandbox dictionary in the converter context of the ll.xist.ns.code namespace.

  • The tests have been ported to py.test.

  • The method mapped() is now callable without arguments. In this case a converter will be created on the fly. You can pass constructor arguments for this converter to mapped() as keyword arguments.

  • The publishing API has changed again: ll.xist.publishers.Publisher.publish() no longer accepts an argument stream to which the byte strings are written, but it is a generator now. The publisher methods write() and writetext() have been renamed to encode() and encodetext() and return the encoded byte string, instead of writing it directly to the stream. There’s a new generator method bytes() for nodes now, which can be passed the same arguments as asBytes(). These changes should help when using XIST in WSGI applications.

  • The iterator returned from Element.__getitem__(), Frag.__getitem__() and the walk() method now supports __getitem__() itself, so you can write table[html.tr][0] to get the first row from a table or page.walk(xsc.FindTypeAll(html.td))[-1] to get the last table cell from a complete HTML page.

  • Several bugs in the namespaces ll.xist.ns.meta, ll.xist.ns.form and ll.xist.ns.specials have been fixed.

  • The namespace modules ll.xist.ns.css and ll.xist.ns.cssspecials have been removed.

Changes in 2.9 (released 2005-04-21)

  • XIST trees can now be pickled. The only restriction is that global attributes must come from a namespace that has been turned into a module via makemod(), so that this module can be imported on unpickling.

  • Two arguments of the walk() method have been renamed: filtermode has been renamed to inmode and walkmode has been renamed to outmode. For these modes two new values are supported:

    ll.xist.xsc.walkindex

    The value passed to the filter function or yielded from the iterator is a list containing child indizes and attribute names that specify the path to the node in question.

    ll.xist.xsc.walkrootindex

    The filter function will be called with two arguments: The first is the root node of the tree (i.e. the node for which walk() has been called), the second one is an index path (just like for ll.xist.xsc.walkindex). If used as an outmode a tuple with these two values will be yielded.

  • Attribute mappings now support __getitem__(), __setitem__() and

    __delitem__() with list arguments, i.e. you can do:

    >>> from ll.xist.ns import html
>>> e = html.a("gurk", href=("hinz", "kunz"))
>>> print e.attrs[["href", 0]]
hinz
>>> e.attrs[["href", 0]] = "hurz"
>>> print e["href"]
hurzkunz
>>> del e.attrs[["href", 0]]
>>> print e["href"]
kunz

    XML attributes can now be accessed as Python attributes, i.e.:

    >>> from ll.xist.ns import html
>>> e = html.a("spam", href="eggs")
>>> print e.attrs.href
eggs

    (Don’t confuse this with e.Attrs.href which is the attribute class.)

  • Frag and Element now support Node subclasses as arguments to their __getitem__() method: An iterator for all children of the specified type will be returned.

  • The encoding used for parsing now defaults to None. When reading from an URL and no default encoding has been specified the one from the Content-Type header will be used. If this still doesn’t result in a usable encoding, "utf-8" will be used when parsing XML and "iso-8859-1" will be used when parsing broken HTML.

  • All error and warning classes from ll.xist.errors have been merged into ll.xist.xsc. This avoids import problems with circular imports.

  • The attributes showLocation and showPath of ll.xist.presenters.TreePresenter have been lowercased and presenters are properly reset after they’ve done their job.

  • The class attribute xmlname will no longer be turned into a list containing the Python and the XML name, but will be the XML name only. You can get the Python name of foo from foo.__class__.__name__.

  • DeprecationWarnings for name and attrHandlers have finally been removed.

  • Instances of ll.xist.xsc.Entity subclasses can now be compared. __eq__() simply checks if the objects are instances of the same class.

Changes in 2.8.1 (released 2005-03-22)

  • Added a note about the package init file to the installation documentation.

Changes in 2.8 (released 2005-01-03)

  • XIST requires Python 2.4 now.

  • ll.xist.ns.specials.x has been renamed to ll.xist.ns.specials.ignore.

  • ll.xist.utils.findAttr() has been renamed to ll.xist.utils.findattr().

  • ll.xist.xfind.item no longer handles slices.

  • XFind has been enhanced to support item and slice operators, i.e. if foo is an XFind operator, foo[0] is an operator that will produce the first node from foo (if there is one). Negative values and slices are supported too.

  • Operators can be chained via division: html.a/html.b is an operator that can be passed around and applied to a node.

  • XIST requires the new core module and makes use of the new “cooperative displayhook” functionality defined there: If you install the displayhook you can tweak or replace ll.xist.presenters.hookpresenter to change the output.

Changes in 2.7 (released 2004-11-24)

  • The transparent pixel used by ll.xist.ns.htmlspecials.pixel has been renamed to spc.gif to avoid problems with IE.

  • Removed a debug print in ll.xist.xfind.Finder.__getitem__.

  • ll.xist.xfind now has a new function item(), that can be used to get a certain item or slice from an iterator. xfind.first() and xfind.last() have been changed to use xfind.item(), so you now have to pass a default value to get the old behaviour.

  • Obsolete options in ll.xist.options have been removed (and reprEncoding has been renamed to reprencoding).

Changes in 2.6.2 (released 2005-06-06)

  • Fixed a bug in ll.xist.parsers.Parser.parse().

Changes in 2.6.1 (released 2004-11-02)

  • Fixed a bug in ll.xist.xfind.Finder.__floordiv__().

  • Restricted characters as defined in XML 1.1 will now be published as character references.

Changes in 2.6 (released 2004-10-26)

  • ToNode() now tries iterating through the value passed in, so it’s now possible to pass iterators and generators (and generator expressions in Python 2.4) to Frag and Element constructors.

  • A new API named XFind has been added for iterating through XML trees. XFind expressions look somewhat like XPath expressions but are pure Python expressions. For example finding all images inside links in an HTML page can be done like this:

    from ll.xist import parsers, xfind
from ll.xist.ns import html
node = parsers.parseURL("http://www.python.org/", tidy=True)
for img in node//html.a/html.img:
   print img["src"]

  • The module ll.xist.xfind contains several operators that can be used in XFind expressions.

  • Parsing broken HTML is now done with the HTML parser from libxml2. The parsing functions no longer accept options for tidy, only the boolean value of the tidy argument is used.

  • The publishing API has been simplified: Publication can now be done with a call to ll.xist.publishers.Publisher.publish(), passing in a ll.xist.xsc.Node. Writing strings to the publisher output is now done with ll.xist.publishers.Publisher.write(). The methods beginPublication() and endPublication() have been removed.

  • The presentation API has been simplified in the same way: You’ll get a presentation by calling: string = presenter.present(node). The methods beginPresentation() and endPresentation() have been removed.

  • The parser now has the option to ignore illegal elements, attributes, processing instructions and entities. The default behaviour is to raise an exception, but this can now be reconfigured via Python’s warning framework.

  • The classmethod tokenize() from ll.toxic has been moved to ll.xist.xsc.Namespace, so it’s now possible to tokenize an XML string for other processing instructions as well.

  • A new class ll.xist.xsc.NSPool has been added. An NSPool contains a pool of namespaces from which the parser selects the appropriate namespace once an xmlns attribute is encountered.</item>

  • The script xscmake.py (which has been unmaintained for a while now) has been removed.</item>

  • Elements hostname, tty, prompt and input were added to ll.xist.ns.doc.

  • The method ll.xist.xsc.Attrs.set() now returns the new attribute object.

  • The visit() method has been removed.

  • ll.xist.xsc.FindOld() has been removed.

  • ll.xist.ns.xml.header has been renamed to ll.xist.ns.xml.declaration.

Changes in 2.5 (released 2004-06-30)

  • Specifying content models for elements has seen major enhancements. The boolean class attribute empty has been replaced by an object model whose checkvalid() method will be called for validating the element content.

  • A new module ll.xist.sims has been added that provides a simple schema validation. Schema violations will be reported via Pythons warning framework.

  • All namespace modules have been updated to use sims information. The SVG module has been updated to SVG 1.1. The docbook module has been updated to DocBook 4.3.

  • It’s possible to switch off validation during parsing and publishing.

  • ll.xist.xsc.Frag and ll.xist.xsc.Element both have a __call__() method with the same arguments as their constructors. Those methods will append content nodes (and set attributes for ll.xist.xsc.Element) and return self, so they can be used when creating an object tree. This makes it possible to put the attributes close to the tag name, instead of putting them at the end after the content.

    Instead of:

    node = html.table(
   html.tr(
      html.td("foo"),
      html.td("bar"),
   ),
   html.tr(
      html.td("spam"),
      html.td("eggs")
   ),
   class_="example"

    you can now use the following:

    node = html.table(class_="example")(
   html.tr(
      html.td("foo"),
      html.td("bar"),
   ),
   html.tr(
      html.td("spam"),
      html.td("eggs")
   )
)

  • Experimental support for Holger Krekel’s XPython has been added. Code might look like this:

    from ll.xist import xsc, converters
from ll.xist.ns import html, meta

import random

c = converters.Converter()
<c>:
   <html.html()>:
      <html.head()>:
         <meta.contenttype()>: pass
         <html.title()>:
            xsc.append("The title")
      <html.body(class_="foo")>:
         <html.h1()>:
            flag = random.choice((0, 1))
            if flag:
               xsc.append("The foo page", class_="foo")
            else:
               xsc.append("The bar page", class_="bar")
         <html.p()>:
            if flag:
               xsc.append("The foo content")
            else:
               xsc.append("The bar content")

print c.lastnode.asBytes()

  • Creating global attributes has been simplified. Passing an instance of ll.xist.xsc.Namespace.Attrs to an Element constructor now does the right thing:

    from ll.xist.ns import html, xml
node = html.html(
   html.head(),
   xml.Attrs(lang="de"),
   lang="en",
)

  • Creating skeleton implementations of XIST namespaces is no longer done via XML conversion (i.e. the namespace module ll.xist.ns.xndl), but through the new module ll.xist.xnd. The script dtdxsc.py will automatically generate sims information.

  • ll.xist.xsc.CharRef now inherits from ll.xist.xsc.Text too, so you don’t have to special case CharRefs any more. When publishing, CharRefs will be handled like Text nodes.

  • ll.xist.ns.meta.contenttype now has an attribute mimetype (defaulting to "text/html") for specifying the MIME type.

  • ll.xist.ns.htmlspecials.caps has been removed.

  • Registering elements in namespace classes has been rewritten to use a cache now.

  • Pretty printing has been changed: Whitespace will only be added now if there are no text nodes in element content.

  • Two mailing lists are now available: One for discussion about XIST and one for XIST announcements.

Changes in 2.4.1 (released 2004-01-05)

  • Changed the xmlname of ll.xist.ns.jsp.directive_page back again (it’s directive.page only for the XML form, which we don’t use anyway.)

  • Drop the default value for ll.xist.ns.jsp.directive_page.Attrs.language, as this attribute can only be used once.

  • If an ll.xist.xsc.Prefixes object has a prefix mapping for a namespace it will return this prefix too, if asked for a prefix for a subclass of this namespace.

Changes in 2.4 (released 2004-01-02)

  • The class ll.xist.parsers.Handler has been renamed to Parser and has been made reusable, i.e. it is possible to instantiate a parser once and use it multiple times for parsing. All the classes derived from xml.sax.xmlreader.InputSource have been dropped and the methods for parsing strings, URLs and files have been implemented as methods of the parser. Most of the arguments that had to be passed to the various parsing functions are passed to the parser constructor now. The basic parsing functionality is implemented by parsing streams instead of InputSource objects.

  • Similar to the changes for parsing, publishers have been changed to be reusable and most arguments to the publishing functions are available as arguments to the publisher constructor.

  • Now converter contexts are no longer bound to an element class, but to the context class defined by the element class, i.e. the attribute Context of the argument for Converter.__getitem__() will be used as the dictionary key. This makes it possible to use a class and it subclasses interchangeably (as long as the base class defines its own Context class and the subclasses don’t overwrite it).

  • Added a find functor FindTypeAllAttrs that searches content and attributes.

  • Fixed the XML name for ll.xist.ns.jsp.directive_page.

  • All character references in ll.xist.ns.ihtml that exist in ll.xist.ns.chars too have been removed.

Changes in 2.3 (released 2003-12-08)

  • It’s now possible to parse XML without generating location information for each node, by passing loc=False to the constructor of the Handler.

  • The HTMLParser no longer complains about global attributes or xmlns.

  • XIST now supports uTidylib in addition to mxTidy. uTidylib is found it is preferred over mxTidy.

  • It’s possible now to pass arguments to tidy simple by passing an argument dictionary for the tidy argument in the parsing functions.

  • The methods parsed() and checkvalid() have been separated.

  • ll.xist.ns.htmlspecials.pixel and ll.xist.ns.htmlspecials.autopixel now check whether their color attribute is ok.

  • The base URL is now set correctly when parsing from an URL even if the original URL does a redirect. (This requires ll.url version 0.11.3).

  • Namespace handling has been rewritten again, to be more standards compliant: Now there is no prefixes for entities and processing instructions any longer. Prefix mappings can be created much simpler, and they no longer contain any namespace stack for parsing, as this is now done by the parser itself. xsc.NamespaceAttrMixIn is gone too.

  • The processing instructions exec_ and eval_ from ll.xist.ns.code have been renamed to pyexec and pyeval and import_ has been removed.

  • CharRefs from ll.xist.ns.html have been moved to a new module named ll.xist.ns.chars.

  • The method names beginPublication(), endPublication() and doPublication() have been lowercased.

Changes in 2.2 (released 2003-07-31)

  • Namespace handling has been completely rewritten. Namespaces are now classes derived from ll.xist.xsc.Namespace. Defining element classes can be done inside or outside the namespace class. If the element classes are defined outside the namespace class, they can be moved inside the namespace with a simple attribute assignment:

    class foo(xsc.Element):
   empty = False

class xmlns(xsc.Namespace):
   xmlname = "foo"
   xmlurl = "http://www.foo.com/ns/foo"

xmlns.foo = foo

  • The methods elementkeys(), iterelementkeys(), elementvalues(), iterelementvalues(), elementitems() and iterelementitems() can be used for iterating through the element classes and their names. You can use the method element() to get an element class with a certain name:

    >>> from ll.xist.ns import html
>>> html.element("div")
<element class ll.xist.ns.html/div at 0x824363c>

  • For processing instructions, entities and character references similar methods are available.

  • The method update() can be used to add many element classes to a namespace at once, simply by passing a dictionary with those classes (use vars() to add everything that’s defined inside your module). The method updatenew() does the same, but copies only those attributes that don’t exist in the namespace, updateexisting() copies only those that do exist. You can turn a namespace into a module with makemod():

    from ll.xist import xsc

class foo(xsc.Element):
   empty = False

class xmlns(xsc.Namespace):
   xmlname = "foo"
   xmlurl = "http://www.foo.com/ns/foo"
xmlns.makemod(vars())

  • Put the above code into foo.py and you can do the following:

    >>> import foo
>>> foo
<namespace foo/xmlns name=u'foo' url=u'http://www.foo.com/ns/foo' with 1 elements from 'foo.py' at 0x81bfc14>

  • getns() has been dropped, so you always have to pass in a Namespace class where a namespace is required.

  • For the ll.xist.ns.jsp.directive_page element automatic generation of the correct charset option in the contentType attribute is only done when there is a contentType attribute, as contentType is optional.

  • The converter has a new property node(). node can’t be passed to conv() but will be set to self by conv() automatically. This makes it possible to access the “document root” during conversion.

  • ll.xist.ns.htmlspecials.autoimg no longer touches existing width and height attributes. This means that %-formatting of the existing attributes is no longer done.

  • Added a new class ll.xist.ns.htmlspecials.autopixel that works like ll.xist.ns.htmlspecials.pixel but inherits the size for the image specified via the src attribute.

  • Frag and Element now support extended slices.

  • Frag and Element now support the methods extend() and __iadd__().

  • For walking the tree the method walk() has been completely rewritten and a new method visit() has been added. For more info see the docstrings.

  • Node now has two new methods copy() and deepcopy() and supports the copy module from the Python standard library.

  • Calling mapped() through conv() has been removed. You again have to call mapped() directly and pass a node and a converter.

  • The HTML handling of the HTMLParser has been improved (it now uses code from xml.sax.drivers2.drv_sgmlop_html (which is part of PyXML.

  • The core functionality found in the script dtd2xsc.py has been moved to a class method ll.xist.ns.xndl.fromdtd() in the ll.xist.ns.xndl namespace.

  • ll.xist.parsers.ExpatParser is now a real subclass instead of an alias for xml.sax.expatreader.ExpatParser It reports unknown entity references to the application (if loading of external entities is switched off, which is done by ll.xist.parsers.Handler and only outside of attributes).

  • Namespaces have been added for Zope’s TAL and METAL specifications.

  • A namespace has been added for XSL-FO.

Changes in 2.1.4 (released 2003-06-13)

  • Remove the checks for attributes in attributes and moved the publication code for the full element into a separate method. This allows JSP tag library namespaces to simply overwrite publish() to publish the element even inside attributes. (This is the same fix as in release 1.5.10).

Changes in 2.1.3 (released 2003-05-07)

  • The methods sorted(), reversed() and shuffled() have been rewritten so they no longer use sys.maxint. This change fixes those methods for 64 bit platforms (reported by Giles Frances Hall)

Changes in 2.1.2 (released 2003-02-27)

  • ll.xist.ns.struts_config11.plug_in now allows content (as the DTD states). (This is the same fix as in release 1.5.8.)

Changes in 2.1.1 (released 2003-02-11)

  • Added a few elements and attributes to ll.xist.ns.doc: username, which is used for the name of a user account, xref, which is used for internal cross references and the attribute id for section, which specifies the target for an xref.

Changes in 2.1 (released 2002-12-09)

  • Added a new namespace module ll.xist.ns.xndl that contains the “XIST namespace definition language”, i.e. elements that describe an XIST namespace and can be used by various scripts to generate skeleton namespace modules. The first of these script is the DTD to namespace converter dtd2xsc.py.

  • Added a new namespace module ll.xist.ns.tld that contains the definition for Java Server Pages Tag Library descriptors and a script tld2xsc.py that uses this namespace to generate namespace modules from tld files.

  • Attr now supports the method filtered(). This is used by without() now. The arguments for without() have changed, because handling global attributes was too “magic”. A new method with() has been added, with does the opposite of without(), i.e. it removes all attributes that are not specified as parameters.

  • The Python name of each Node subclass is now available as the class attribute pyname.

  • To continue the great renaming withSep() has been renamed to withsep().

  • The namespace name for the ll.xist.ns.struts_html module has been fixed.

  • The argument defaultEncoding for the various parsing functions has been renamed to encoding.

Changes in 2.0.8 (released 2002-11-20)

  • ll.xist.ns.doc.getDoc() has been renamed to getdoc().

  • The CSS parser was dropping the % from percentage values. This has been fixed.

Changes in 2.0.7 (released 2002-11-12)

  • xsc.Element.__nonzero__() can no longer fall back to xsc.Frag.__nonzero__(). (this is the same fix as in 1.5.7).

Changes in 2.0.6 (released 2002-11-11)

  • Performance optimizations.

Changes in 2.0.5 (released 2002-11-11)

  • Fixed a bug in ll.xist.ns.specials.autoimg: Attributes were not converted before the size check was done (this is the same fix as in 1.5.5).

Changes in 2.0.4 (released 2002-11-08)

  • Fixed a regression bug in ll.xist.ns.jsp.directive and several documentation issues.

Changes in 2.0.3 (released 2002-10-30)

  • Fixed a few bugs in HTMLParser.

  • Added DocBook conversion for several elements in ll.xist.ns.doc.

  • Now the __init__.py file for the ll package is included.

Changes in 2.0.2 (released 2002-10-21)

  • Fixed a bug in Frag.__rmul__() (by reusing __mul__()).

  • Fixed a bug with the backwards compatible prefix mapping: Defining element classes in exec processing instructions didn’t work, because the prefixes object used for parsing wouldn’t be updated when the namespace object is defined inside the processing instruction. Now using the default for the prefixes argument in calls to the parsing functions uses one global shared Prefixes instances where all the namespaces that are newly defined will be registered too.

Changes in 2.0.1 (released 2002-10-17)

  • Fixed xscmake.py by removing the prefix handling. OldPrefixes will always be used for parsing now.

Changes in 2.0 (released 2002-10-16)

  • XIST now requires at least Python 2.2.1.

  • Attribute handling has been largely rewritten. Instead of a class attribute attrHandlers, the attributes are now defined through a nested class named Attrs inside the element. This class must be derived from ll.xist.Element.Attrs (or one of its subclasses if you want to inherit attributes from this class). Defining attributes is done through classes nested inside this attributes class and derived from any of the known attribute classes (like TextAttr, URLAttr etc.). The class name will be the attribute name (and can be overwritten with a class attribute xmlname. This makes it possible to have docstrings for attributes. Furthermore it’s possible to define an attribute default value via the class attribute default, allowed values for the attribute via values, which is a list of allowed values, and whether the attribute is required or not via required.

  • XIST now has real namespace support. The new class ll.xist.xsc.Prefixes can be used to define a mapping between prefixes and namespace names. This can be used for parsing and publishing. Namespace support is even available for entities and processing instruction.

  • Global attributes are supported now. Namespace modules for the xml and xlink namespaces have been added (and ll.xist.xsc.XML was moved to ll.xist.ns.xml).

  • A new namespace module for SVG 1.0 has been added: ll.xist.ns.svg.

  • The HTML specific parts of ll.xist.ns.specials have been split off into a separate module ll.xist.ns.htmlspecials.

  • Comparison of attributes with strings has been removed. You have to use __unicode__() or __str__() now before comparing.

  • The HTMLParser now removes unknown attributes instead of complaining.

  • There is a new parser class BadEntityParser, which is a SAX2 parser that recognizes the character entities defined in HTML and tries to pass on unknown or malformed entities to the handler literally.

  • To give all nodes a chance to do something after they have been parsed (e.g. to prepend the base URL for URLAttr nodes), the parser now calls the method parsed() immediately after node creation. This is used for the new class StyleAttr, which uses the CSSTokenizer, to prepend the base URL to all URLs found in a style attribute.

  • The pixel images have been moved to the directory px to make image URLs shorter.

Changes in 1.6.1 (released 2003-08-25)

  • Updated to work with newer versions of ll.ansistyle.

  • Updated the namespaces ll.xist.ns.struts_html and ll.xist.ns.struts_config11 to the state of Struts 1.1 final.

Changes in 1.6 (released 2003-07-02)

  • Removed the default value for the className attribute in ll.xist.ns.struts_config11.action.

  • Added an attribute type to ll.xist.ns.struts_config11.action_mapping.

Changes in 1.5.13 (released 2003-07-01)

  • Implemented ll.xist.xsc.Namespace.__eq__(), so that replacing a namespace in the registry really works.

  • Added an attribute target to ll.xist.ns.html.area.

Changes in 1.5.12 (released 2003-06-17)

Changes in 1.5.11 (released 2003-06-13)

Changes in 1.5.10 (released 2003-06-13)

  • Remove the checks for attributes in attributes and moved the publication code for the full element into a separate method. This allows JSP tag library namespaces to simply overwrite publish() to publish the element even inside attributes.

Changes in 1.5.9 (released 2003-04-30)

  • Reregistering a namespace now properly overwrites the old version in xsc.namespaceRegistry.

Changes in 1.5.8 (released 2003-02-27)

  • ll.xist.ns.struts_config11.plug_in now allows content (as the DTD states).

Changes in 1.5.7 (released 2002-11-12)

  • xsc.Element.__nonzero__() can no longer fall back to xsc.Frag.__nonzero__().

Changes in 1.5.6 (released 2002-11-11)

  • Performance optimizations.

Changes in 1.5.5 (released 2002-11-11)

  • Fixed a bug in ll.xist.ns.specials.autoimg: Attributes were not converted before the size check was done.

Changes in 1.5.4 (released 2002-09-30)

  • xscmake.py now tries to strip off a trailing xsc from the filename before it falls back to the extension html (The builtin extension mapping is still tried first).

Changes in 1.5.3 (released 2002-09-25)

  • Added new processing instruction class ll.xist.ns.php.expression that generates a PHP print statement from its content.

Changes in 1.5.2 (released 2002-09-19)

  • Removed the value magic from ll.xist.ns.form.checkbox as this conflicted with dynamic value values.

Changes in 1.5.1 (released 2002-09-17)

  • Comparison of attributes with strings has been removed. You have to use __unicode__() or __str__() instead.

  • The HTMLParser now removes unknown attributes instead of complaining.

  • There is a new parser class BadEntityParser, which is a SAX2 parser that recognizes the character entities defined in HTML and tries to pass on unknown or malformed entities to the handler literally.

  • To give all nodes a chance to do something after they have been parsed (e.g. to prepend the base URL for URLAttr nodes), the parser now calls the method parsed() immediately after node creation. This is used for the new class StyleAttr, which uses the CSSTokenizer, to prepend the base url to all urls found in a style attribute.

  • The HTMLParser now removes unknown attributes instead of complaining.

  • There is a new parser class BadEntityParser, which is a SAX2 parser that recognizes the character entities defined in HTML and tries to pass on unknown or malformed entities to the handler literally.

  • To give all nodes a chance to do something after they have been parsed (e.g. to prepend the base URL for URLAttr nodes), the parser now calls the method parsed() immediately after node creation. This is used for the new class StyleAttr, which uses the CSSTokenizer, to prepend to base URL to all URLs found in a style attribute.

Changes in 1.4.3 (released 2002-04-29)

  • New namespace module xist.ns.struts_config11 allows to parse and modify Struts configuration files conforming to the Struts 1.1 DTD.

Changes in 1.4.2 (released 2002-03-22)

  • Updated xscmake.py to be compatible with the new url module.

  • xist.ns.jsp.directive_page now automatically sets the contentType on publishing.

Changes in 1.4.1 (released 2002-03-21)

  • Removed TidyURLInputSource. Now it’s possible to pass a tidy flag to the remaining functions parseString(), parseFile() and parseURL() to specify whether the source should be tidied.

  • To prevent an element from being registered in a Namespace the class attribute register can be used now. This makes it possible to have a name for the element even when it’s not registered.

  • xist.ns.form elements now have all the attributes that the corresponding elements from xist.ns.html have.

  • Removed the old xist.url from the Windows distribution.

Changes in 1.4 (released 2002-03-18)

  • Reimplemented URL handling again. Now the new global module url is used for that.

Changes in 1.3.1 (released 2002-03-14)

  • Added a method pretty() to Node for generating a pretty printable version of the node.

  • xsc.Node.name no longer is a class method, but a class attribute, that will be set at class instantiation time by the meta class.

Changes in 1.3 (released 2002-02-12)

  • Ported to Python 2.2. Node is now derived from object, Frag from list and there’s a new class Attrs which is derived from dict for the attribute mappings. All presenters have been adapted to work with Attrs. In addition to the usual dictionary methods and operators Attrs has a method without() that returns a copy of the Attrs instance with some specified attributes removed.

  • All the node classes now have a new method walk() that generates all nodes in the tree using the new generator feature of Python 2.2.

  • Also a new method walkPath() has been added that works the same as walk() but yields the complete path to each node as a list.

  • Added a class block to xist.ns.jsp. The content of the block instance will simply be enclosed in a {} block. xist.ns.php got such a class too.

  • Added a new module xist.ns.ihtml for i-mode HTML.

  • Added new modules xist.ns.css and xist.ns.cssspecials for generating CSS.

  • Now the various attributes of the Converter object are collected in a ConverterState object and it’s possible to push and pop those states, i.e. it’s now easy to temporarily modify a converter object during a convert() call and revert back to a previous state afterwards.

  • parseURL() and parseTidyURL() now have an additional parameter headers which is a list of string pairs specifying additional headers to be passed in with the request.

  • parseString() has an additional parameter systemId which will be the system id of the InputSource.

  • The distribution now includes the makefile and the XML source files so now the distribution can rebuild ifself.

  • Various other small bugfixes and enhancements.

Changes in 1.2.5 (released 2001-12-03)

  • Added a new element contentscripttype to xist.ns.meta that generates a <meta http-equiv="Content-Script-Type" ...> element.

  • xist.ns.doc.explain() now generates anchor elements for the class, function and method description, so now the links on the XIST webpages work.

  • Docstrings and documentation has been reworked. Now xist.ns.doc.pyref no longer implies a font change. Use the classes xist.ns.doc.module, xist.ns.doc.class, xist.ns.doc.method, xist.ns.doc.function and xist.ns.doc.arg to mark up your Python identifiers.

  • Added the attributes type and key to xist.ns.struts_config.data_source.

Changes in 1.2.4 (released 2001-11-23)

  • Added the deprecated attributes start to xist.ns.html.ol and value to xist.ns.html.li.

Changes in 1.2.3 (released 2001-11-22)

  • Added missing asPlainString() methods to Comment and DocType.

Changes in 1.2.2 (released 2001-11-16)

  • xist.url.URL.fileSize() and xist.url.URL.imageSize() now use the warning framework to report errors.

  • There is a new presenter named CodePresenter that dumps the tree as Python source code.

  • The filenames of the pixel images used by xist.ns.specials.pixel have changed. These images are now included.

Changes in 1.2.1 (released 2001-10-08)

  • URLs that are completely dynamic will now be left in peace when parsing or publishing.

Changes in 1.2 (released 2001-10-03)

  • xist.ns.meta.keywords and xist.ns.meta.description no longer call asPlainString() on their content. This makes it possible to e.g. generate the keywords via JSP:

    >>> from xist import parsers
>>> from xist.ns import meta, jsp
>>> s = '<keywords>' + \
...     '<?jsp:expression "foo"?>' + \
...     '</keywords>'
>>> e = parsers.parseString(s)
>>> print e.conv().asBytes()
<meta name="keywords" content="<%= "foo" %>" />

  • When an element occurs inside an attribute during publishing, there won’t be an exception raised any more. Instead the content of the element will be published. This fixes problems with abbreviation entities inside attributes.

  • xist.parsers.TidyURLInputSource now uses the new experimental eGenix mx Extension package, which includes a Python port of tidy.

  • __repr__() now uses the new class presenters.PlainPresenter which gives a little more info than the default __repr__().

  • URL handling has been changed again. Upto now, URLAttr had an additional instance attribute base, which was the “base” file/URL from which the attribute was parsed. Now the base URL will be directly incorporated into the URL. You can pass the base URL to all the parsing functions. Similar to that when publishing you can specify a base URL. All URLs in the tree will be output relative to this base URL. Joining URLs is now done via __div__() and no longer via __add__(). This makes it more consistent with fileutils. The plan is to make URLs string like immutable objects and to merge them with fileutils.Filename.

  • xist.ns.specials.php has been moved to its own module (xist.ns.php). This module provided additional convenience processing instructions (just like xist.ns.jsp does).

Changes in 1.1.3 (released 2001-09-17)

  • The global namespace registry now keeps a sequential list of all registered namespaces, which will be used by the parser when searching for names. This gives a predictable search order even without using Namespaces and its pushNamespace() method: modules imported last will be searched first.

  • Processing instructions are now allowed inside attributes when publishing.

  • xist.ns.docbooklite has been renamed to xist.ns.doc. It can now generate HTML and Docbook output and has improved a lot. The XIST web pages now use this for automatic documentation generation. The doc example has been removed.

  • xist.url.URL now has a new method info() that returns the headers for the file/URL.

  • xist.url.URL now has a methods fileSize() and imageSize() too.

  • xist.ns.jsp.directive_page now has new attribute session.

Changes in 1.1.2 (released 2001-08-21)

  • __repr__() now uses the new class presenters.PlainPresenter which gives a little more info than the default __repr__().

Changes in 1.1.1 (released 2001-08-01)

  • Small bugfix in presenters.strProcInst().

  • Fixed xist.ns.struts_html.option to allow content.

Changes in 1.1 (released 2001-07-19)

  • Sequences in constructor arguments for Frag and Element are again expanded and it’s again possible to pass dictionaries in an Element constructor to specify attributes. As sequences are always unpacked, the method extend() is gone. This works for append() and insert() too.

  • Node and Frag implement __mul__() and __rmul__(), so you can do stuff like:

    html.br()*5

    This returns a Frag with five times to same node.

  • Arguments for the converter constructor can be passed to xist.xsc.Node.conv() now, so it’s possible to do stuff like this:

    from xist.ns import code
print code.Eval("return converter.lang").conv(lang="en").asBytes()

    which will print en.

  • The option XHTML for the publishers has been changed to lowercase.

  • xist.ns.html.html will automatically generate a lang and xml:lang attribute when the converter has a language set.

Changes in 1.0 (released 2001-06-18)

  • New module for WML 1.3.

  • The publishing interface has changed internally and publishing should be faster now.

  • Publishers now support a new parameter: usePrefix, which specifies if namespace prefixes should be output for the element names.

  • Part of the implementation of the publishing stuff has been moved to C, so now you’ll need a C compiler to install XIST.

  • When publishing ", it will now only be replaced with &quot; inside attributes.

  • All the asHTML() methods now have an additional argument converter. This makes it possible to implement different processing modes or stages for new elements. All currently implemented elements and entities ignore this argument, but pass it on in the call to their childrens’ asHTML() method. As the name asHTML() no longer makes sense, asHTML() has been renamed to convert().

  • There is now a tool dtd2xsc.py in the scripts directory that creates a skeleton XIST module from a DTD (this requires xmlproc from the PyXML package).

  • New preliminary module for DocBook 4.12. (Incomplete: convert() methods and Unicode character entities are missing; any volunteers for implementing 375 classes?)

  • New module ruby.py that implements the W3C Ruby draft.

  • sql.py has been removed from XIST, but is available as a separate module.

  • The parsing interface has been changed. Parsing is now done with the functions parseFile(), parseString(), parseURL() and parseTidyURL() in the module parsers. It’s now possible to specify which parser should be used for parsing by passing a SAX2 parser instance to any of these functions. XIST now includes a rudimentary SAX2 driver for sgmlop and a rudimentary HTML parser that emits SAX2 events.

  • The python-quotes example has been updated to work with expat.

  • Added a new example: media.

  • All abbreviation entities have been moved to a new module abbr.py.

  • All the modules that provide new elements and entitites have been moved to a subpackage ns.

  • Frag and Element now have new methods sorted(), reversed(), filtered() and shuffled() that return sorted, reversed, filtered and shuffled versions of the Frag/Element object.

  • New namespace modules ns/jsp.py and ns/struts_html.py have been added that allow you to use JSP and Struts tags with XIST.

  • A new method asText() was added, that returns the node as a formatted plain ASCII text (this requires that w3m is installed.)

  • make.py has been renamed to xscmake.py and moved to the scripts directory, it will be installed as a callable script with python setup.py install_scripts.

  • xscmake.py has a new option --files/-f. The argument is a file containing a list of filenames (one name per line) that should be converted.

  • xscmake.py has a new option --parser/-r for specifying which parser to use. Allowed values are sgmlop and expat.

  • xscmake.py has a new option --namespace/-n that can be used for appending Namespace objects to the Namespaces object used by xscmake.py:

    xscmake.py -n html -n spam eggs.xsc

    With this call the parser will find element classes from the module with the prefix name spam before those from html and those before anything else.

  • xist.url.URL no longer has an attribute ext. file and ext are merged.

  • The special treatment of sequences as constructor arguments to Frag and Element has been removed, so XIST will no longer remove one level of nesting. If you still want that, use a * argument.

  • Frag and Element now have a new method mapped(), that recursively maps the nodes through a function. This is like convert() but via an external function.

  • Attribute handling has been improved thanks to a suggestion by Hartmut Goebel: Element.__getitem__() now always works as long as the attribute name is legal. If the attribute is not set, an empty attribute will be returned. All empty attributes will be considered as being not set and so hasAttr() returns 0 for them, and publish() doesn’t publish them. This simplifies several very common cases:

    • Copying an attribute from one element to another works regardless of whether the attribute is set or not;

    • Testing for an attributes presence can now be done much simpler: if element["attrname"] instead of if element.hasAttr("attrname") (which still works, and should be a little faster);

    • When you construct an XIST tree and the presence or absence of an attribute is tied to a condition, you can construct the attribute in advance and use it afterwards in the tree construction:

      if condition:
   align = "right"
else:
   align = None
node = html.div("spam", align=align)

      So, when the condition is false, the node will not have the attribute align set.

  • xist.ns.cond.If (and xist.ns.cond.ElIf) can now be used to test for attributes of the converter. I.e. it’s possible to write the following XML:

    <if lang="en">Title
<elif lang="de">Überschrift
</if>

  • URL handling has be completely changed and is much, much simpler now. There are no more path markers. To specify an URL that is relative to the current directory use the scheme root (e.g. root:main.css).

Changes in 0.4.7 (released 2000-11-24)

  • Fixed a bug in the entity handling.

  • Added a few deprecated elements and attributes to the html module.

  • Improved the publishing of attributes. Now all attribute values will be published. For boolean attributes no value will be published for XHTML==0 and the attribute name will be used for XHTML==1 or XHTML==2.

  • Element.compact() now works (better) ;).

  • Incorparated many bug fixes from Hartmut Goebel.

  • Implemented xsc.Element.copyDefaultAttrs(), which copies unset attributes over from a dictionary (simplifies implementing specials.plaintable and specials.plainbody).

  • providers.Provider.pushNamespace() now handles multiple arguments which may be Namespace objects or modules (in which case, module.namespace will be pushed).

  • providers.Providers.popNamespace() can now pop multiple namespaces at once.

  • providers.TidyURIProvider now uses os.popen3() for piping the file through tidy, so now there will be no more temporary files. The call to tidy now includes options that hopefully make the output more suited to XIST.

  • Incorparated a new url.py by Hartmut Goebel, that fixes many problem (e.g. optimizing http://server/foo/bar/../../baz.gif now works.)

  • make.py includes a new option --path for adding directories to sys.path.

Changes in 0.4.6 (released 2000-11-03)

  • Now uses sgmlop.XMLParser instead of sgmlop.SGMLParser, so case is preserved.

  • Fixed another regression from the URL to string conversion change.

Changes in 0.4.5 (released 2000-11-01)

  • Converting URLs to nodes is now done in ToNode(), so URL objects can be used everywhere.

  • Fixed a few bugs in Text._strtext() and URLAttr._str().

Changes in 0.4.4 (releases 10/27/2000)

  • Now testing if characters can be encoded with the specified encoding is done directy. This means, that escaping unencodable characters now works even with exotic encodings (tested with JapaneseCodecs 1.0.1.

  • The URLAttr constructor now can handle a single parameter of the type URL.

  • The URL to string conversion function have changed: URL.asString() returns the URL with path markers, URL.asPlainString() returns the URL without path markers.

  • Added the i18n attribute to the font element.

  • Fixed the clashes between the class names for the elements and entities sub and sup in html.py.

  • Several small enhancements and bug fixes contributed by Hartmut Goebel.

Changes in 0.4.3 (released 2000-10-19)

  • Now processing instruction classes are registered in the same way as elements and entities are.

  • The leaf nodes (Text, Comment, ProcInst) are now considered immutable. This means that their asHTML() method can simply return self, because now those nodes can be shared between trees. Functionality for manipulation the objects is provided by a mixin class very similar to UserString. All this results in a speedup of about 10% for the python-quotes example.

  • Small optimizations in the asHTML() methods of Element and Frag optimized away many calls to append(), extend() and ToNode() and result in a speedup of about 30% for the python-quotes example. One consequence of this is that Null objects will no longer be ignored.

Changes in 0.4.2 (released 2000-09-24)

  • New elements keywords and description in meta.py.

  • Fixed a bug in Namespace.register(), now setting name=None to prevent an element from being registered works again.

Changes in 0.4.1 (released 2000-09-21)

  • A new module named meta.py has been created, that simplifies generating meta tags.

  • Various small bugfixes.

Changes in 0.4 (released 2000-09-19)

  • XIST now requires at least Python 2.0b1.

  • A new bugfixed version of the sgmlop source is available from the FTP site.

  • XIST now completely supports Unicode. For output any encoding known to Python can be used, so now you can output your HTML in ASCII, Latin-1, UTF-8, UTF-16, …

  • All publishers have been updated to support Unicode. The publishing interface has been streamlined (encoding and XHTML parameters are now attributes of the publisher).

  • asString() will now always return a Unicode string. If you want a byte string use asBytes() instead, where the encoding can be specified as an argument.

  • There an additional publisher class FilePublisher, which can be used for publishing to a file (or anything else that has a write() and a writelines() method, and is supported by the stream writer available through codecs.lookup()).

  • Element and attribute names are no longer converted to lowercase. If you have an attribute name which clashes with a Python keyword (e.g. class) append an underscore (_), which will be removed before accessing the attribute. This is the “official” Python method for handling these cases.

  • Elements and entities are no longer registered one by one. Now you can build Namespace objects which are used for searching and there are pushNamespace() and popNamespace() functions in XSC.xsc. For more info, see the source.

  • Image size calculation has been removed from html.img and html.input. Use specials.autoimg and specials.autoinput for that.

  • __getitem__(), __setitem__() and __delitem() of Frag and Element now accepts a list as an argument. The method will be applied recursively, i.e. e[[0, 1, "foo", 2] is the same as e[0][1]["foo"][2].

  • The deprecated module db.py no longer exists. Useful functions and elements from db.py have been moved to sql.py and form.py respectively.

  • When using xsc.make() the encoding and XHTML parameters to use can now be specified on the command line (e.g. --encoding utf-8 --xhtml 2)

  • Handling of multiline <?xsc-eval?> and <?xsc-exec?> has been enhanced, although XIST will not be able to guess the correct indentation in all cases. As a workarround simply add a Python comment to the beginning. So the following won’t work:

    <?xsc-exec
   for i in xrange(10):
      do(i)
?>

    But this will:

    <?xsc-exec
   #
   for i in xrange(10):
      do(i)
?>

  • Make functionality has been moved to make.py, as certain modules can’t be used as the main script, because reimporting them in processing instructions won’t work. Now you can simply call:

    make.py --import xist.html --import spam eggs.xsc

  • There is a new module cond.py, that contains elements that can be used for conditionals:

    <?xsc-exec a=42?>
<if cond="a==21">
   <b>foo</b>
<elif cond="a==42"/>
   <i>bar</i>
<else/>
   baz
</if>

Changes in 0.3.9 (released 2000-08-10)

  • sgmlop will now be found either via import sgmlop or via from xml.parsers import sgmlop.

Changes in 0.3.8 (released 2000-07-14)

  • Fixed a bug in URLAttr.publish(), which prevented URLAttr from working at all.

Changes in 0.3.7 (released 2000-07-06)

  • Fixed a bug in html.img and html.input. Now image size calculation works again.

  • Changes in 0.3.6 (released 2000-07-04)

  • Fixed a bug in Node._matches(), which resulted in a non working find().

Changes in 0.3.5 (released 2000-07-02)

  • The documentation example has been enhanced. Now documenting methods works.

  • When the member elementname: in the element class is set before calling registerElement(), this element name will be used for the element. This allows custom names even when using registerAllElements().

  • Comparison of scheme and server in URLs is done case insensitive (as RFC 2068 requires.)

  • Image size calculation is now done in asString() and not in asHTML(). This allows to write faster code. Old method:

    e = html.div(html.img(...),gurk.hurz()).asHTML().asString()

    New method:

    e = html.div(html.img(...),gurk.hurz().asHTML()).asString()

  • Image size calculation is now done for <input type="image">. The size attribute is set to the image width.

  • Manipulating the path in an URL is now done via the usual __setitem__()/__getitem__() stuff, which keeps the path in a consistent state:

    >>> from xist.URL import URL
>>> u = URL("/foo/*/../bar/baz.gif")
>>> del u[1]
>>> u
URL(scheme='server', path=['bar'], file='baz', ext='gif')

  • findNodes() (which has been shortened to find()) has an additional argument test, which can be a test function that will be called when the node passes all other tests.

  • asString() no longer generates a string directly, but uses the new method publish(), which has an additional argument publisher, to which the strings to be output are passed.

Changes in 0.3.4 (released 2000-05-31)

  • Location information is now copied over in clone(), asHTML() and compact() where appropriate, so you know even in the HTML tree where something came from.

  • xsc.repransi can now have three values:

    0

    coloring is off

    1

    coloring is on for a dark background

    2

    coloring is on for a light background

  • All repransi variables are now arrays with two strings, the first for dark, the second for light.

Changes in 0.3.3 (released 2000-05-30)

  • The workaround for the trailing CDATA bug in sgmlop has been removed, so now you’ll need a newer version of sgmlop (included in PyXML 0.5.5.1).

Changes before 0.3.3

  • These changes predate written history.