Migration

This document describes how to update your code to each version of XIST. Only incompatible changes are listed here. For a list of all changes see History.

Migrating to version 5.76

Changes to ll.ul4c

  • The output infrastructure for UL4 (which was using generator before) has been switched to a stream API. The new API is incompatible with the old one.

Migrating to version 5.73

Changes to ll.pysql

  • Some PySQL commands have been renamed: resetsequence to reset_sequence, checkerrors to check_errors, raiseexceptions to raise_exceptions, pushraiseexceptions to push_raise_exceptions and popraiseexceptions to pop_raise_exceptions.

  • The argument raiseexceptions to various PySQL commands has been renamed to raise_exceptions.

Migrating to version 5.67

Changes to ll.orasql

Migrating to version 5.66

Changes to ll.color

  • The color method abslum() has been renamed to abslight() and the method rellum() has been renamed to rellight().

Changes to UL4

  • UL4 functions and methods have been updated to use positional-only or keyword-only arguments to match the signature of the corresponding Python function/method.

  • Subclasses of ll.ul4c.AST have been renamed so that their name matches the name of the corresponding class in the Java implementation. (for example ll.ul4c.Add has been renamed to ll.ul4c.AddAST)

  • The UL4 function type() now returns type objects instead of simple strings. To get the name of the type use the type objects __name__ attribute, i.e. replace type(foo) with type(foo).__name__.

  • Naming of attributes that are used to implement UL4 functionality is more uniform now. This affects the following attributes: The methods ul4_getattr(), ul4_setattr() and ul4_hasattr() for implementing object attribute access from UL4; the methods ul4_call(), ul4_render() and , ul4_renders() for making objects callable or renderable from UL4; the class attribute ul4_attrs for exposing a number of readonly attributes to UL4; the attributes ul4_context that is used for marking a callable as needing the context as an argument in the call.

  • Support for using custom tag delimiters for UL4 templates tag has been removed, i.e. now the tag delimiters will always be <? and ?>.

  • The color method abslum() has been renamed to abslight() and the method rellum() has been renamed to rellight().

Changes to ll.misc and ll.ul4on

  • Some function now use positional-only arguments:

    • ll.misc.item(iterable, index, /, default=None)

    • ll.misc.first(iterable, /, default=None)

    • ll.misc.last(iterable, /, default=None)

    • ll.misc.count(iterable, /)

    • ll.misc.isfirst(iterable, /)

    • ll.misc.islast(iterable, /)

    • ll.misc.isfirstlast(iterable, /)

    • ll.misc.monthdelta.__init__(self, months=0, /)

    • ll.ul4on.dumps(obj, /, indent)

    • ll.ul4on.dump(obj, /, stream, indent)

    • ll.ul4on.load(stream, /, registry=None)

    • ll.ul4on.loads(dump, /, registry=None)

    • ll.ul4on.loadclob(clob, /, bufsize=1024*1024, registry=None)

    Update your calls accordingly.

Migrating to version 5.58

Changes to ll.sisyphus

  • The method healthcheck() should no longer be implemented. Instead the option --maxhealthcheckage or the class/instance attribute maxhealthcheckage can be used to configure the maximum allowed age.

  • The method failed() is no longer supported. If you need its functionality wrap the content of your execute() method in a try block and put the content of you failed() method into an except block (and reraise the exception at the end of the except block).

  • The filenames for log files can no longer be changed via options or job attributes, instead one of the following methods must be overwritten:

    • basedir()

    • logfilename()

    • currentloglinkname()

    • lastsuccessfulloglinkname()

    • lastfailedloglinkname()

    • lastinterruptedloglinkname()

    • lasttimeoutloglinkname()

    • healthfilename()

    • emailfilename()

    Those methods must return an absolute path as a pathlib.Path object.

Migrating to version 5.57

Changes to ll.ul4on

ll.ul4on.Encoder and ll.ul4on.Decoder now expect the stream to be passed in the call to dump() and load() instead of in the constructor. I.e. change:

Encoder(stream).dump(obj)

to:

Encoder().dump(obj, stream)

and:

obj = Decoder(stream).load()

to:

obj = Decoder().load(stream)

The parameter name for the UL4ON function fromul4on() has changed from string to dump.

Migrating to version 5.56

Changes to ll.orasql

ll.orasql.Comment has been renamed to ll.orasql.ColumnComment.

Migrating to version 5.52

Changes to ll.orasql

The method getobject() for ll.orasql.Synonym has been renamed to object().

Migrating to version 5.45

Changes to UL4

The UL4 AST node attribute line and col have been renamed to startline and startcol.

Migrating to version 5.44

Changes to ll.pysql

The PySQL command compileall has been removed. This same effect can simply be achieved by calling utl_recomp.recomp_parallel() or dbms_utility.compile_schema().

The PySQL terminator comment (-- @@@) can now no longer be specified via a command line option.

The connectstring argument for the pysql script is now optional, so it has to be specified via the optional argument -d/--database.

The --commit argument (with the options record, once and never) has been replaced with a flag option --rollback. Automatically committing after every record is no longer available. However manual committing is available via the commit command.

PySQL no longer supports multiple active database connections via the connectname key. When using literal SQL this couldn’t be used anyway, so it has been dropped. If you really need this feature you can implement a workaround in literal Python blocks.

Migrating to version 5.42

Changes to ll.sisyphus

Returning None from ll.sisyphus.Job.execute() now has a special meaning: Delete the log file. If this isn’t wanted a string (e.g. "done") should be returned.

Migrating to version 5.40

Changes to ll.orasql

The class ll.orasql.SchemaObject has been renamed to ll.orasql.OwnedSchemaObject. (ll.orasql.SchemaObject now is for objects that don’t have an owner (i.e. ll.orasql.User and ll.orasql.JobClass).)

Migrating to version 5.37

Changes to UL4

Exception chaining has changed, so the exception you get from calling/rendering an UL4 template now is the original exception. To get to the information about the location in the UL4 source code you have to iterate through the __cause__ chain.

The interal structure of UL4 templates has been simplified, but that should only concern you if you’ve worked with the UL4 AST itself. Basically Tag objects are gone now, and instead each AST node has attributes template (referencing the outermost template) and pos (being the position of the nodes source code). This also means that blocks no longer have an endtag attribute.

Migrating to version 5.36

Changes to ll.orasql

As cx_Oracle provides its own Object orasql.Object has been renamed to orasql.SchemaObject.

Migrating to version 5.34

Changes to XIST

The class ll.xist.ns.html.script.Attrs.async has beend renamed to async_, because async is a keyword in Python 3.7.

Migrating to version 5.32

Changes to ll.orasql

The default value for the owner parameter in various ll.orasql methods has changed from ALL to None (i.e. it now returns the objects from the current schema instead of all schemas).

The changed methods are: Connection.tables(), Connection.sequences(), Connection.fks(), Connection.privileges(), Connection.objects(), Object.names(), Object.objects() (and all subclasses of Object) and Privilege.objects().

To get the old behaviour back, simply pass owner=orasql.ALL to those methods.

Migrating to version 5.28

Changes to UL4

  • UL4 now longer tries a disguise objects as dictionaries. I.e. for objects with an ul4attrs class attribute the methods items, keys, values and get are no longer synthesized. This also means that len, list, item access and containment tests no longer work on objects. However iterating over the attribute names of an object can now be done with the new function dir. To get, set and test attributes, the new functions getattr, setattr and hasattr can be used.

Migrating to version 5.22

Changes to pysql

  • The values for the option -v/--verbose has changed: -v1 now is -vdot, -v2 is -vtype and -v3 is -vfull.

Migrating to version 5.21

Changes to ll.color

  • Colors can no longer be added. This was done with the formula:

    0.5*(c1.r+c2.r), 0.5*(c1.g+c2.g), 0.5*(c1.b+c2.b), 255-(255-c1.a)*(255-c2.a)/255.)

Changes to ll.orasql

  • The method ll.orasql.ForeignKey.pk() has been renamed to refconstraint().

Migrating to version 5.20

Changes to ul4

  • The variables passed to UL4 templates in rul4 have been moved into a globals objects. The following changes have to be made to the template source:

    • change oracle.connect(...) to globals.oracle(...);

    • change mysql.connect(...) to globals.mysql(...);

    • change sqlite.connect(...) to globals.sqlite(...);

    • change system.execute(...) to globals.system(...);

    • change load(...) to globals.load(...);

    • change error(...) to globals.error(...);

    • change foo to globals.vars.foo for a variable foo defined via rul4 -D.

Migrating to version 5.18

Changes to ul4

  • The UL4 exception ll.ul4c.Error has been renamed to LocationError.

  • The UL4 function type now returns the Python class name for date, color, template exception objects.

Migrating to version 5.17

Changes to rul4

The function import has been split into load for loading the content of a file and compile for compiling a string, so:

<?code template = import("/home/user/template/foo.ul4")?>

has to be replaced with:

<?code template = compile(load("/home/user/template/foo.ul4"))?>

Migrating to version 5.16

Changes to orasql

Some methods in orasql have been renamed: Iterating methods no longer have iter in their name (e.g. itertables() is now simply called tables()). The ddl part of some method names has been changed to sql (e.g. createddl() is now called createsql()).

Migrating to version 5.15

Changes to PySQL

  • The function load has been replaced by two functions loadstr for loading strings and loadbytes for loading bytes, i.e. replace:

    load('foo.txt', 'utf-8', 'replace')

    with:

    loadstr('foo.txt', 'utf-8', 'replace')

    and:

    load('foo.png')

    with:

    loadbytes('foo.png')

  • PySQL no longer supports the -- !!! command terminator. Use the raiseexceptions command instead to specify error handling.

Migrating to version 5.14

Changes to UL4

  • The boolean parameter keepws for ul4c.Template has been renamed to whitespace and requires a string value now. Pass whitespace="keep" for the old keepws=True and whitespace="strip" for the old keepws=False.

  • The rul4 option --keepws has been renamed to --whitespace and defaults to smart now. So instead of the old --keepws=1 pass --whitespace=keep and for --keepws=0 pass --whitespace=strip.

  • Rendering an UL4 template from inside a UL4 template is now again done via the <?render?> tag. So inside a template you have to replace the code:

    <?code template.render(foo, bar)?>

    with:

    <?render template(foo, bar)?>

  • Closures in UL4 templates no longer see the state of the variables at the time when the local template was defined, but at the time when it is called. This is similar to most other languages that support closures.

    To emulate the old behaviour pass the variables you want to “freeze” to a locally defined template and define the original template there.

Changes to pysql

  • SQL commands must be terminated with a -- @@@ (or -- !!!) comment line now, i.e. now the comment after the command determines whether exceptions will be ignored, instead of the comment before the command.

Migrating to version 5.13

Changes to UL4

  • Locally defined UL4 templates no longer see themselves among the variables of the parent template.

Changes to sisyphus

  • The option setproctitle for sisyphus jobs has been renamed to proctitle.

  • The default for the name parameter in tasks() for sisyphus jobs has changed from str to None, i.e. it defaults to unnamed tasks now.

Migrating to version 5.12

Changes to ul4on

  • The UL4ON serialization format has been reimplemented to be more human-readable and robust. The new format is incompatible to the old. If you update your XIST installation to 5.12 you should update the corresponding UL4ON versions for Java/Javascript too.

Migrating to version 5.10

Changes to misc

  • The functions misc.gzip() and misc.gunzip() have been removed as Python 3.2 has the functions gzip.compress() and gzip.uncompress(), which do the same.

Migrating to version 5.9

Changes to db2ul4

  • The script db2ul4 has been renamed to rul4.

Changes to ll.url

  • The argument pattern of the URL methods listdir(), files(), dirs(), walk(), walkfiles() and walkdirs() has been renamed to include.

  • The method walk() has been renamed to walkall().

Migrating to version 5.7

Changes to ll.oradd

  • The file command has been renamed to scp.

Changes to ll.orasql

  • The methods ll.orasql.Record.keys() and ll.orasql.Record.values() return iterators now. ll.orasql.Record.iterkeys() and ll.orasql.Record.itervalues() have been removed.

Migrating to version 5.6

Changes to ll.oradd

  • Support for "keys" and "sqls" has been removed from ll.oradd. So

    {
   "type": "procedure",
   "name": "procname",
   "args": {
      "proc_id": "p_10",
      "proc_date": "sysdate",
      "keys": {"proc_id": "int"},
      "sqls": ["proc_date"]
   }
}

    has to be replaced with

    {
   "type": "procedure",
   "name": "procname",
   "args": {
      "proc_id": var("p_10", int),
      "proc_date": sql("sysdate")
   }
}

  • UL4ON dumps are no longer supported by ll.oradd. They must be reencoded as Python repr outputs, which can be done with code that looks like this:

    import sys

from ll import ul4on

while True:
   try:
      print(repr(ul4on.load(sys.stdin)))
   except EOFError:
      break

Migrating to version 5.4

Changes to ll.url

  • The remotepython parameter for ssh URLs has been renamed to python.

Migrating to version 5.2

Changes to sisyphus

  • The method prefix() for sisyphus jobs has been replaced with task() which does something similar.

Changes to UL4

  • The names of methods that should be callable for custom objects in UL4 templates must be added to the ul4attrs attributes.

Changes to oradd

  • Committing the transactions in oradd can now be done after each record with the new option --commit. --rollback has been removed, so you have to replace --rollback=1 with --commit=never.

Changes to misc

  • The default argument for the functions misc.first() and misc.last() now defaults to None. I.e. for empty iterators the default value will always be returned instead of generating an exception. To simulate the old behaviour use a unique guard object as the default.

  • Renamed the attributes scriptname and shortscriptname of the misc.sysinfo object to script_name and short_script_name.

Migrating to version 5.1

Changes to db2ul4

  • The query method for database connections has changed: Instead of a query and a parameter dictionary, you have to pass in positional arguments that alternate between fragments of the SQL query and parameters. I.e.:

    db.query("select * from table where x=:x and y=:y", x=23, y=42)

    becomes:

    db.query("select * from table where x=", 23, " and y=", 42)

    This makes db2ul4 independent from the parameter format of the database driver.

Migrating to version 5.0

Changes to XIST

  • Accessing attributes via __getattr__(), __setattr__() and __delattr__() now requires the XML name of the attribute instead of the Python name. If you only have the Python name, you can convert it to the XML name with the method Attrs._pyname2xmlname().

  • For all methods that existed in Python/XML pairs (e.g. withnames() and withnames_xml() in xsc.Attrs or elementclass() and elementclass_xml() in xsc.Pool etc.) there is only one version now: A method without the _xml suffix in the name, that accepts the XML version of the name.

  • Validation is now off by default, to turn it on pass validate=True to parse.tree() or parse.itertree() for parsing, or to the publisher object or the bytes(), iterbytes(), string() or iterstring() methods for publishing.

Migrating to version 4.10

Changes to UL4

  • The UL4 tag <?render?> have been removed. To update your code replace <?render r.render()?> with <?exe r.render()?>.

  • The UL4 functions vars and get have been removed.

  • The automatic UL4 variable stack has been removed too.

Migrating to version 4.7

Changes to UL4

  • Compiling a UL4 template to a Java CompiledTemplate is no longer supported (i.e. template.javasource(interpreted=False) no longer works. Use template.javasource() instead (which creates Java sourcecode for an InterpretedTemplate).

Migrating to version 4.6

Changes to ll.xist

  • The walk() method has been changed to return a Cursor object instead of the path, so you have to replace:

    for path in doc.walk(...):
   # use path

    with:

    for cursor in doc.walk(...):
   # use cursor.path

  • Furthermore walk filters have been removed. Determining whether an XIST tree is traversed top down or bottom up can instead by specified via distinct parameters to the walk() method. Replace:

    for path in doc.walk((xfind.entercontent, xfind.enterattrs, True)):
   ...

    with:

    for cursor in doc.walk(entercontent=True, enterattrs=True, startelementnode=False, endelementnode=True):
   ...

    If you want to enter an element only when a condition is true, you can do that by modifying the appropriate cursor attribute inside your loop:

    for cursor in doc.walk(entercontent=True, enterattrs=True):
   if isinstance(cursor.node, html.script, html.textarea):
      cursor.entercontent = False
   ...

  • ll.xist.parse.itertree() now returns Cursor objects too, instead of path lists.

  • Slicing XIST elements now returns a sliced element, instead of a slice from the content Frag:

    >>> from ll.xist.ns import html
>>> html.ul(html.li(i) for i in range(5))[1:3].string()
'<ul><li>1</li><li>2</li></ul>'

    To get a slice from the content simply use:

    >>> html.ul(html.li(i) for i in range(5)).content[1:3].string()
'<li>1</li><li>2</li>'

Migrating to version 4.4

Changes to the required Python version

Python 3.3 is required now.

Migrating to version 4.2

Changes to ll.ul4c

  • The UL4 method join no longer calls str on the items in the argument list. Replace sep.join(iterable) with sep.join(str(i) for i in iterable) when you have an argument list that contains non-strings.

Migrating to version 4.1

Changes to ll.make

  • The support for Growl notifications in ll.make on the Mac has been replaced by support for Mountain Lions Notification Center.

    The option has been renamed from --growl to --notify.

    For this to work you need to have terminal-notifier installed in its standard location (/Applications/terminal-notifier.app).

Migrating to version 4.0

Changes to the required Python version

Python 3.2 is required now.

Changes to UL4

  • Date constants in UL4 have changed again. They are now written like this: @(2012-04-12) or @(2012-04-12T12:34:56).

  • The function json has been renamed to asjson.

  • The <?render?> tag in UL4 now looks like a method call instead of a function call. I.e. <?render t(a=17, b=23)?> has changed to <?render t.render(a=17, b=23)?>.

Changes to scripts

  • The scripts oracreate, oradrop, oradelete, oradiff, oramerge, oragrant, orafind and uhpp no longer have an -e/--encoding option. They always use Pythons output encoding.

  • The options -i/--inputencoding and -o/--outputencoding of the script db2ul4 have been replaced with an option -e/--encoding for the encoding of the template files. For printing the result Pythons output encoding is used.

  • The options --inputencoding/--inputerrors and --outputencoding/--outputerrors of ll.sisyphus.Job have been replaced with option --encoding/--errors for the encoding of the log files.

Migrating to version 3.25

Changes to XIST

  • The compact() method has been renamed to compacted() to avoid collisions with the compact attribute in HTML elements.

Migrating to version 3.24

Changes to ll.xist.ns.ul4

  • ll.xist.ns.ul4.attr_if is now an ll.xist.xsc.AttrElement subclass. Change your code from:

    html.div(id=(ul4.attr_if("foo"), "bar"))

    to:

    html.div(id=ul4.attr_if("bar", cond="foo"))

  • ll.xist.ns.ul4.attr_ifnn has been removed. Replace it with the equivalent attr_if call.

Migrating to version 3.23

Changes to ll.ul4c

  • The module global functions ll.ul4c.compile(), ll.ul4c.load() and ll.ul4c.loads() have been removed. Instead of them the Template constructor and the class methods load() and loads() can be used.

Migrating to version 3.20

Changes to ll.orasql

  • The schema argument used by various methods in ll.orasql has been replaced by a owner argument that can be None (for the current user), the constant ALL for all users (which uses the DBA_* variant of various meta data views if possible or the ALL_* variants otherwise) and a specific user name.

Migrating to version 3.19

Changes to ll.orasql

  • ll.orasql now requires cx_Oracle 5.1 (i.e. UNICODE mode is no longer used).

  • If the readlobs option is false for ll.orasql cursors, the CLOBs/BLOBs returned will be wrapped into something that behaves like a Python file. The original LOB object is available as the value attribute of the returned wrapper object:

    db = orasql.connect("user/pwd@db")
c = db.cursor()
c.execute("select theclob from thetable")
row = c.fetchone()
print row[0].value.read()

Migrating to version 3.18

Changes to db2ul4

  • The variables available in UL4 templates used by db2ul4 have changed. Instead of a connect object, there are now three objects for each supported database (i.e. oracle, sqlite and mysql). To update your template replace:

    connect["oracle:user/pwd@db"]

    with:

    oracle["user/pwd@db"]

Changes to scripts

  • The script doc2txt now reads from stdin and writes to stdout instead of requiring file names on the command line.

Migrating to version 3.17

Changes to ll.misc

  • ll.misc.javastring() has been renamed to ll.misc.javaexpr().

  • The UL4 method format is now a function instead.

Migrating to version 3.16

Changes to ll.misc

  • ll.misc.flag() is gone. If the function is still required, here is the source:

    def flag(value):
   if value in ("1", "true", "yes"):
      return True
   elif value in ("0", "false", "no"):
      return False
   raise ValueError("unknown flag value")

Migrating to version 3.15

Changes to ll.xist.ns.jsp

  • ll.xist.ns.jsp.javastring() has been move to ll.misc.

Migrating to version 3.14

Changes to ll.ul4c

  • Date constants now need a @ as a prefix. I.e. chance 2010-11-03T to @2010-11-03T etc.

  • The function argument for ul4c.Template.pythonsource() is gone. The output will always be a full function.

Migrating to version 3.12

Changes to ll.sisyphus

  • The maximum allowed runtime for jobs is now a hard limit. Previously a running job that exceeded the maximum allowed runtime would only be killed when the next job was started. Now the job will kill itself immediately after maxtime seconds. This means you might have to adjust your maxtime setting.

  • The default location of log files has changed again. Now ~/ll.sisyphus/ is used as the base directory instead of ~/ll.sisyphus/log/.

Migrating to version 3.11

Changes to ll.sisyphus

  • The method logLoop() is gone. Replace:

    self.logLoop("done")

    with:

    return "done"

  • The method logProgress() is gone. Replace:

    self.logProgress("parsing XML file")

    with:

    self.log("parsing XML file")

    You might also add tags to the logging call via:

    self.log.xml("parsing XML")

    (This adds the tag "xml" to the log line.)

  • The method logError() is gone. Replace:

    self.logError("Can't parse XML file")

    with:

    self.log.error("Can't parse XML file")

    If the object passed to self.log is an exception, the logging call will add the exc tag automatically.

  • sisyphus.Job no longer has a constructor. Configuration is now done via class attributes. Replace:

    class TransmogrifyStuff(sisyphus.Job):
   def __init__(self, connectstring):
      sisyphus.Job.__init__(self, 30, "ACME_TransmogrifyStuff", raiseerrors=True)

    with:

    class TransmogrifyStuff(sisyphus.Job):
   projectname = "ACME.MyProject"
   jobname = "TransmogrifyStuff"
   maxtime = 30

  • The default location of run/log files has changed. Now ~/ll.sisyphus/log is used for log files and ~/ll.sisyphus/run is used for run files.

Migrating to version 3.10

Changes to the required Python version

Python 2.7 is required now.

Changes to ll.make

Changes to ll.daemon

  • ll.daemon uses argparse now. ll.daemon.Daemon.optionparser() has been renamed to argparser().

Migrating to version 3.9

Changes to ll.xist.ns.html

Migrating to version 3.8

Changes to parsing

  • The parsing infrastructure has been completely rewritten to be more modular and to support iterative parsing (similar to ElementTree). Now parsing XML is done in a pipeline approach.

    Previously parsing a string looked like this:

    >>> from ll.xist import xsc, parsers
>>> from ll.xist.ns import html
>>> source = "<a href='http://www.python.org/'>Python</a>"
>>> doc = parsers.parsestring(source, pool=xsc.Pool(html))

    Now this is done the following way:

    >>> from ll.xist import xsc, parse
>>> from ll.xist.ns import html
>>> source = "<a href='http://www.python.org/'>Python</a>"
>>> doc = parse.tree(
...   parse.String(source)
...   parse.Expat()
...   parse.NS(html)
...   parse.Node(pool=xsc.Pool(html))
... )

    For more info see the module ll.xist.parse.

  • Something that no longer works is parsing XML where elements from different namespaces use the same namespace prefix. You will either have to rewrite your XML or implement a new class for the parsing pipeline that handles namespaces prefixes and instantiating XIST classes (i.e. a combination of what ll.xist.parse.NS and ll.xist.parse.Node do).

  • The module ll.xist.parsers has been renamed to parse.

  • The module ll.xist.presenters has been renamed to present.

  • The classes ll.xist.converters.Converter and ll.xist.publishers.Publisher have been moved to ll.xist.xsc. The modules ll.xist.converters and ll.xist.publishers no longer exist.

Changes to XISTs walk filters

  • The walk methods walknode() and walkpath() have been renamed to walknodes() and walkpaths(). The class WalkFilter has been moved to ll.xist.xfind.

Changes to ll.url

  • ll.url.Path has been simplified: Path segments are strings instead of tuples. If you need the path parameters (i.e. part after ; in a path segment) you have to split the segment yourself.

  • ll.url.URL.import_() is gone. As a replacement misc.module() can be used, i.e. replace:

    >>> from ll import url
>>> u = url.File("foo.py")
>>> m = u.import_(mode="always")

    with:

    >>> from ll import url, misc
>>> u = url.File("foo.py")
>>> m = misc.module(u.openread().read(), u.local())

    However, note that ll.url.URL.import_() has been reintroduced in 3.8.1 based on misc.import(). This means that the mode argument is no longer supported.

  • ssh URLs now required to standalone execnet package. The ssh_config parameter for ssh URLs is gone.

Changes to ll.make

  • The two classes ll.make.PoolAction and ll.make.XISTPoolAction have been dropped. To update your code, replace:

    make.XISTPoolAction(html)

    with:

    make.ObjectAction(xsc.Pool).call(html)

  • The class XISTParseAction has been removed. This action can be replaced by a combination of ObjectAction, CallAction and CallAttrAction using the new parsing infrastructure.

Other changes

Migrating to version 3.7

Changes to the make module

  • The division operator for actions is no longer implemented, so instead of:

    t1 = make.FileAction(key=url.URL("file:foo.txt"))
t2 = t1 /
     make.DecodeAction("iso-8859-1") /
     make.EncodeAction("utf-8") /
     make.FileAction(key=url.URL("bar.txt"))

    you now have to write something like the following:

    t1 = make.FileAction("file:foo.txt")
t2 = t1.callattr("decode", "iso-8859-1")
t2 = t2.callattr("encode", "utf-8")
t2 = make.FileAction("file:bar.txt", t2)

  • Also the following classes have been removed from ll.make: EncodeAction, DecodeAction, EvalAction, GZipAction, GUnzipAction, JavascriptMinifyAction, XISTBytesAction, XISTStringAction, JoinAction, UnpickleAction, PickleAction, TOXICAction, TOXICPrettifyAction, SplatAction, UL4CompileAction, UL4RenderAction, UL4DumpAction, UL4LoadAction, XISTTextAction and XISTConvertAction. All of these actions can be executed by using CallAction or CallAttrAction.

Migrating to version 3.6

Changes to the color module

  • The following Color class methods have been dropped: fromrgba, fromrgba4, fromrgba8, fromint4, fromint8.

  • The following Color properties have been dropped: r4, g4, b4, a4, r8, g8, b8, a8, r, g, b, a int4, int8, rgb4, rgba4, rgb8, and rgba8. The new methods r, g, b and a return the 8 bit component values.

  • The class methods fromhsva and fromhlsa have been renamed to fromhsv and fromhls.

  • The property css has been dropped. The CSS string is returned by __str__ now.

  • Dividing colors now does a scalar division. Blending colors is now done with the modulo operator.

Removal of XPIT

  • The XPIT templating language has been removed. You should replace all your XPIT templates with UL4 templates.

Migrating to version 3.5

Changes to UL4

  • The UL4 function csvescape has been renamed to csv.

Changes to the color module

  • ll.color.Color has been rewritten to create immutable objects with the components being 8 bit values (i.e. 0-255) instead of floating point values between 0 and 1.

Migrating to version 3.4

Changes to the make module

  • ll.make.CallMethAction has been renamed to CallAttrAction.

  • ll.make.XISTPublishAction has been renamed to XISTBytesAction.

Changes to UL4

  • The templates available to the <?render?> tag are no longer passed as a separate argument to the render methods, but can be part of the normal variables.

Changes to XIST

  • Building trees with with blocks has changed slightly. Unchanged code will lead to the following exception:

    File "/usr/local/lib/python2.5/site-packages/ll/xist/xsc.py", line 1285, in __enter__
   threadlocalnodehandler.handler.enter(self)
AttributeError: 'NoneType' object has no attribute 'enter'

    To fix this, change your code from:

    with html.html() as node:
   with html.head():
      +html.title("Foo")
   with html.body():
      +html.p("The foo page!")

    to:

    with xsc.build():
   with html.html() as node:
      with html.head():
         +html.title("Foo")
      with html.body():
         +html.p("The foo page!")

    (i.e. wrap the outermost with block in another with xsc.build() block.)

Migrating to version 3.3

Changes to the make module

  • ll.make.ImportAction has been dropped as now the module object can be used directly (e.g. as the input for an XISTPoolAction object).

  • The constructor of most action classes now accept the input action as a parameter again. This means that you might have to change the calls. Usually it’s safest to use keyword arguments. I.e. change:

    make.FileAction(url.File("foo.txt"))

    to:

    make.FileAction(key=url.File("foo.txt"))

  • The targetroot parameter for ll.make.XISTConvertAction.__init__() has been renamed to root.

Changes to TOXIC

  • TOXIC has been split into a compiler and an XIST namespace module. Instead of calling the function ll.xist.ns.toxic.xml2ora() you now have to use ll.toxicc.compile(). (However using TOXIC with ll.make hasn’t changed).

Changes to XIST

  • The default parser for XIST is expat now. To switch back to sgmlop simply pass an SGMLOPParser object to the parsing functions:

    >>> from ll.xist import parsers
>>> node = parsers.parsestring("<a>", parser=parsers.SGMLOPParser())

Migrating to version 3.2.6

Changes to escaping

The functions ll.xist.helpers.escapetext and ll.xist.helpers.escapeattr have been merged into ll.misc.xmlescape and all the characters <, >, &, " and ' are escaped now.

Migrating to version 3.1

Changes to URL handling

URLs containing processing instructions will no longer be transformed in any way. If you need the old behaviour you can wrap the initial part of the attribute value into a specials.url PI.

Migrating to version 3.0

Changes to tree traversal

You can no longer apply xfind expression directly to nodes, so instead of:

for node in root//html.p:
   print node

you have to write:

for node in root.walknode(html.p):
   print node

If you want the search anchored at the root node, you can do the following:

for node in root.walknode(root/html.p):
   print node

This will yield html.p elements only if they are immediate children of the root node.

Passing a callable to the walk() method now creates a ll.xist.xfind.CallableSelector. If you want the old tree traversal logic back, you have to put your code into the filterpath() method of a WalkFilter object.

Many of the XFind operators have been renamed (and all have been rewritten). See the xfind documentation for more info.

The death of namespace modules

It’s no longer possible to turn modules into namespaces. Element classes belong to a namespace (in the XML sense) simply if their xmlns attribute have the same value. So a module definition like this:

from ll.xist import xsc

class foo(xsc.Element):
   def convert(self, converter):
      return xsc.Text("foo")

class xmlns(xsc.Namespace):
   xmlname = "foo"
   xmlurl = "http://xmlns.example.org/foo"
xmlns.makemod(vars())

has to be changed into this:

from ll.xist import xsc

class foo(xsc.Element):
   xmlns = "http://xmlns.example.org/foo"

   def convert(self, converter):
      return xsc.Text("foo")

Renamed doc classes

Many classes in the ll.xist.ns.doc module have been renamed. The following names have changed:

  • function to func;

  • method to meth;

  • module to mod;

  • property to prop;

  • title to h;

  • par to p;

  • olist to ol;

  • ulist to ul;

  • dlist to dl;

  • item to li or dd (depending on whether it’s inside an ol, ul or dl);

  • term to dt;

  • link to a.

Migrating to version 2.15

Changes to plain text conversion

The node method asText() has been moved to the html namespace, so you have to replace:

print node.asText()

with:

from ll.xist.ns import html
print html.astext(node)

Changes to htmlspecials.pixel

If you’ve been using the color attribute for htmlspecials.pixel, you have to add a # in from of the value, as it is a CSS color value now. (And if’ve you’ve been using color and a CSS padding of a different color: This will no longer work).

Migrating to version 2.14

Changes to presenters

Presenters work differently now. Instead of:

print node.asrepr(presenters.CodePresenter)

simply do the following:

print presenters.CodePresenter(node)

Migrating to version 2.13

Changes to ll.xist.xsc

xsc.Namespace.tokenize() no longer has an encoding argument, but operates on a unicode string directly. You can either use the result of a asString() call or decode the result of an asBytes() call yourself.

Migrating to version 2.11

Changes to ll.xist.xsc

The function ToNode() has been renamed to tonode().

ll.xist.Context no longer subclasses list. If you need a stack for your context, simply add the list as an attribute of the context object.

Code rearrangements

The iterator stuff from ll.xist.xfind has been moved to the ll package/module, i.e. you have to use ll.first() instead of ll.xist.xfind.first().

Changes to the walk() method

The walk() method has changed again. There are no inmodes and outmodes any longer. Instead input and output are Cursor objects. If you’re using your own walk() filters, you have to update them. For different output modes you can use the methods walknode(), walkpath() or walkindex() instead of using the cursor yielded by walk().

The node methods find() and findfirst() have been removed. Use xsc.Frag(node.walk(...)) or node.walk(...)[0] instead.

Changes to publishing

Publishing has changed: If you’ve used the method repr() before to get a string representation of an XML tree, you have to use asrepr() instead now (repr() is a generator which will produce the string in pieces).

Changes to the xfind module

The functions item(), first(), last(), count() and iterone() as well as the class Iterator have been moved to the ll module.

Migrating to version 2.10

Changes to publishing

Publishing has been changed from using a stream API to using a iterator API. If you’ve been using Publisher.write() or Publisher.writetext() (in your own publish() methods) you must update your code by replacing publisher.write(foo) with yield publisher.encode(foo) and publisher.writetext(foo) with yield publisher.encodetext(foo).

Changes to the test suite

The test suite now uses py.test, so if you want to run it you’ll need py.test.

Changes to ll.xist.ns.code

The code in a ll.xist.ns.code.pyexec object is no longer executed at construction time, but at conversion time. So if you relied on this fact (e.g. to make a namespace available for parsing of the rest of the XML file) you will have to change your code.

Removed namespaces

The namespace modules ll.xist.ns.css and ll.xist.ns.cssspecials have been removed.

Migrating to version 2.9

Changes to exceptions

All exception classes have been moved from ll.xist.errors to ll.xist.xsc.

Changes to XML name handling

The class attribute xmlname no longer gets replaced with a tuple containing both the Python and the XML name. If you want to get the Python name, use foo.__class__.__name__.

Changes to the methods walk(), find() and findfirst()

The argument filtermode has been renamed to inmode and (for walk()) walkmode has been renamed to outmode.

Migrating to version 2.8

Changes to display hooks

The way XIST uses sys.displayhook() has been enhanced. To make use of this, you might want to update your Python startup script. For more info see the installation instructions.

Changes to the xmlns attribute

Each element (or entity, or processing instruction) class had an attribute xmlns that references the namespace module. This attribute has been renamed to __ns__.

Other minor changes

ll.xist.ns.specials.x has been renamed to ll.xist.ns.specials.ignore.

ll.xist.xfind.item no longer handles slices. If you’ve used that functionality, you may now use slices on XFind operators, and materialize the result, i.e. replace xfind.slice(foo, 1, -1) with list(foo[1:-1]), if foo is an XFind operator. Otherwise you can use list(foo)[1:-1].

Migrating to version 2.7

Changes to ll.xist.xfind

The functions xfind.first() and xfind.last() now use xfind.item(), so they will raise an IndexError when no default value is passed. To get the old behaviour, simply pass None as the default.

Migrating to version 2.6

Changes to the publishing API

The top level publishing method in the publisher has been renamed from dopublication() to publish(). If you’re using the publishing API directly (instead of the node methods asBytes() and write()), you’ll have to update your code.

The method that writes a unicode object to the output stream has been renamed from publish() to write(). This is only relevant when you’ve overwritten the publish() method in your own node class (e.g. in JSP tag library directives or similar stuff, or for special nodes that publish some text literally).

Changes to the presentation API

The presentation API has been changed too: The top level presentation method in the presenter has been renamed from dopresentation() to present(). This is only relevant if you’ve written your own presenter, or are using the presentation API directly (instead of the node method repr()).

Parsing HTML

Parsing HTML is now done via libxml2’s HTML parser, instead of using µTidyLib of mxTidy. You can no longer pass arguments to tidy. Only the boolean values of the tidy argument will be used. There are no other visible changes to the API but the result of parsing might have changed.

Removed APIs and scripts

The script xscmake.py has been removed.

The visit() method has been removed.

ll.xist.xsc.FindOld() has been removed.

ll.xist.ns.xml.header has been renamed to ll.xist.ns.xml.declaration.

Migrating to version 2.5

Changes to content model

The boolean class attribute empty for element classes has been replaced by an object model. empty is still supported, but issues a PendingDeprecationWarning. If you don’t want to specify a proper content model for your own elements you can replace empty = False with model = True (which is a shortcut for model = sims.Any()) and empty = True with model = False (which is a shortcut for model = sims.Empty()).

Migrating to version 2.4

Changes to parsing

Parsing has changed internally, but the module level parsing functions in ll.xist.parsers are still available (and will create a parser on the fly), but a few arguments have changed:

handler

This argument is no longer available, if you need a special handler, you have to subclass ll.xist.parsers.Parser and call its parsing methods.

parser

This argument has been renamed to saxparser and is not a SAX2 parser instance any longer, but a callable that will create a SAX2 parser.

sysid

sysid is now available for all parsing functions not just parseString().

Changes to converter contexts

ll.xist.converters.Converter.__getitem__() now doesn’t use the key passed in, but key.Context as the real dictionary key. This has the following consequences:

  • If you want a unique context for your own element class, you must implement a new Context class (otherwise you’d get ll.xist.xsc.Element.Context):

    class Foo(xsc.Element):
   empty = False

   class Context(xsc.Element.Context):
      def __init_(self):
         xsc.Element.Context.__init__(self)
         ...

  • Subclasses that don’t overwrite Context (as well as instances of those classes) can be passed to ll.xist.converters.Converter.__getitem__() and the unique base class context object will be returned.

Changed namespaces

The character reference classes from ll.xist.ns.ihtml that are duplicates of those in ll.xist.ns.chars have been removed, so you have to use ll.xist.ns.chars for those characters in addition to ll.xist.ns.ihtml

Migrating to version 2.3

Changes in namespace handling

Namespace handling has changed. There are no entity or processing instruction prefixes any longer and creating a proper Prefixes object has been simplified. For example:

prefixes = xsc.Prefixes()
prefixes.addElementPrefixMapping(None, html)
prefixes.addElementPrefixMapping("svg", svg)

can be simplified to:

prefixes = xsc.Prefixes(html, svg=svg)

The three arguments elementmode, entitymode and procinstmode for the publishing methods have been combined into prefixmode, which is used for elements only.

Changed namespaces

The character reference classes from ll.xist.ns.html have been moved to a separate namespace ll.xist.ns.chars.

The processing instructions eval_ and exec_ from the ll.xist.ns.code module have been renamed to pyeval and pyexec.

Changed method names

The method names beginPublication(), endPublication() and doPublication() have been lowercased.

Migrating to version 2.2

Attribute methods

The Element methods for accessing attributes have been deprecated. So instead of node.hasattr("attr"), you should use:

"attr" in node.attrs

The same holds for checking whether an attribute is allowed. You can use the following code:

"attr" in node.Attrs

or:

"attr" in NodeClass.Attrs

or:

NodeClass.isallowed("attr")

Many Attrs methods have gained an additional parameter xml, which specifies whether an attribute name should be treated as the XML or the Python name of the attribute. Make sure that you’re not mixing up your arguments in the function call. The safest method for this is using keyword arguments, e.g.:

node.attr.get("attr", default=42)

JSP directive page element

A contentType attribute is no longer generated for the ll.xist.ns.jsp.directive_page. You have to explicitly use an attribute contentType="text/html" to get a contentType attribute in the resulting JSP. The charset option is generated automatically from the encoding specified in the publisher.

autoimg changes

ll.xist.htmlspecials.autoimg will no longer touch existing width or height` attributes, so e.g. setting the width to twice the image size via width="2*%(width)s" no longer works. You have to implement your own version of autoimg if you need this.

find() changes

find() has been completely rewritten to use the new tree traversal filters. For backwards compatibility a filter functor ll.xist.xsc.FindOld exists that takes the same arguments as the old find() method. I.e. you can replace:

node.find(
   type=html.a,
   attr={"href": None},
   searchchildren=True
)

with:

node.find(
   xsc.FindOld(
      type=html.a,
      attr={"href": None},
      searchchildren=True
   ),
   skiproot=True
)

But one minor difference remains: when skiproot is set to true in the new find() method, the attributes of the root element will not be traversed. With the old method they would be traversed.

doc changes

programlisting has been renamed to prog.

Namespace changes

Namespaces can no longer be instantiated. Instead you have to derive a class from Namespace. The xmlprefix argument from the constructor becomes a class attribute xmlname and the argument xmlname becomes xmlurl.

Adding element classes to the namespace is now done with the Namespace classmethod update(). If you want the turn a namespace into a module, you can use the classmethod makemod() instead of update(), i.e. replace:

xmlns = xsc.Namespace("foo", "http://www.foo.com/", vars())

with:

class xmlns(xsc.Namespace):
   xmlname = "foo"
   xmlurl = "http://www.foo.com/"
xmlns.makemod(vars())

Migrating to version 2.1

The method withSep() has been renamed to withsep().

The argument defaultEncoding for the various parsing functions has been renamed to encoding.

Migrating to version 2.0

Attribute handling

The biggest change is in the way attributes are defined. In older versions you had to define a class attribute attrHandlers that mapped attribute names to attribute classes. This created problems with “illegal” attribute names (e.g. class and http-equiv in HTML), so for them an ugly workaround was implemented. With 2.0 this is no longer neccessary. Defining attributes is done via a class Attrs nested inside the element class like this:

class foo(xsc.Element):
   class Attrs(xsc.Element.Attrs):
      class bar(xsc.TextAttr)
         "The bar attribute"
         default = "spam"
         values = ("spam", "eggs")
         required = True
      class baz(xsc.URLAttr):
         "The baz attribute"

Default values, set of allowed attributes values and whether the attribute is required can be defined via class attributes as shown above. You should (directly or indirecty) inherit from xsc.Element.Attrs, because this class implements handling of global attributes. If you want to inherit some attributes (e.g. from your base class), you can derive from the appropriate Attrs class. Removing an attribute you inherited can be done like this:

class bar(foo):
   class Attrs(foo.Attrs):
      baz = None

This removes the attribute baz inherited from foo.

For attribute names that are no legal Python identifiers, the same method can be used as for element classes: Define the real XML name via a class attribute. This class attribute has been renamed from name to xmlname.

This also means that you always have to use the Python name when using attributes now. The XML name will only be used for parsing and publishing.

XIST 2.0 tries to be as backwards compatible as possible: An existing attrHandlers attribute will be converted to an Attrs class on the fly (and will generate a DeprecationWarning when the class is created). An Attrs class will automatically generate an attrHandlers attribute, so it’s possible to derive from new element classes in the old way. The only situation where this won’t work, is with attributes where the Python and XML name differ, you have to use “new style” attributes there.

Namespace support

XIST supports XML namespaces now and for parsing it’s possible to configure which namespaces should be available for instantiating classes from. For more info about this refer to the documentation for the class Prefixes.

Before 2.0 the XML name for a namespace object was pretty useless, now it can be used as the namespace name in xmlns attributes and it will be used for that when publishing and specifying an elementmode of 2 in the call to the publishing method or the constructor of the publisher.

Namespace objects should now be named xmlns instead of namespace as before.

Global attributes

Global attributes are supported now, e.g. the attributes xml:lang and xml:space can be specified in an element constructor like this:

from ll.xist import xsc
from ll.xist.ns import html, xml

node = html.html(
   content,
   {(xml, "lang"): "en", (xml, "space"): "preserve"},
   lang="en"
)

Instead of the module object (which must contain a namespace object named xmlns), you can also pass the namespace object itself (i.e. xml.xmlns) or the namespace name (i.e. "http://www.w3.org/XML/1998/namespace").

Namespace changes

The classes XML and XML10 have been moved from ll.xist.xsc to ll.xist.ns.xml.

All the classes in ll.xist.ns.specials that are specific to HTML generation have been moved to the new module ll.xist.ns.htmlspecials.

The module ll.xist.ns.html has been updated to the XHTML specification, so there might be some changes. The new feature for specifying attribute restrictions has been used, so e.g. you’ll get warnings for missing alt attributes in img elements. These warnings are issued via the warning framework. Refer to the documentation for the warnings module to find out how to configure the handling of these warnings.

Miscellaneous

XIST now requires at least Python 2.2.1 because the integer constants True and False are used throughout the code wherever appropriate. These constants will become instances of the new class bool in Python 2.3. You might want to change your code too, to use these new constant (e.g. when setting the element class attribute empty).

Using mixed case method names was a bad idea, because this conflicts with Python’s convention of using all lowercase names (without underscores). These method names will be fixed in the next few XIST versions. The first names that where changed were the element methods getAttr() and hasAttr(), which have been renamed to getattr() and hasattr() respectively. getAttr() and hasAttr() are still there and can be called without generating deprecation warnings, but they will start to generate warnings in the upcoming versions.