Boost C++ Libraries

In quickbook 1.5 if you include a file which starts with a docinfo block, it's ignored and the file is expanded in place. In quickbook 1.6 it's treated as a document nested in the current position. So if it has an 'article' docinfo block, boostbook 'article' tags are used.

It also mostly generates the same markup as if the file was converted separately - so for example, the same ids are generated, the document is processed using the language version specified in the docinfo block. If no language is specified it uses the default (1.1) not the version of the document that included it. This might seem surprising, but is requried so that quickbook will convert it the same way as if it was converted separately.

So for the most part, includes with a docinfo are like an xinclude, apart from a couple of differences. Templates and macros defined in the parent document are used in the included document, and the id generator rewrites ids that clash between multiple documents.

If an included document doesn't have a docinfo block, it's just included as before.

You can now expand macros in text fields in the docinfo block. In the top docinfo block only the predefined macros are available, but in nested documents macros defined in the parent document are also available.

There's a small bug here - this leaks into older versions for the license and purpose fields, but since only the predefined macros are available, it's unlikely to break any existing documents. So I'd rather not complicate the code further by fixing that.

A long standing quickbook bug is that macros are scoped by file, but templates aren't. So you can define templates in a separate file and include them, but not macros. This has been fixed so that templates defined in one file won't 'leak' into another.

But this means there's no way to define templates in a separate file - a useful feature. To do this the import element has been adapted to also support quickbook files. If a quickbook file is imported, the templates and macros defined in it are added to the current scope, but nothing else contained in that file is used. This could be used to create template and macro library files. This matches the existing semantics of importing code snippets.

When importing templates, they're bound to the language version of the file they were defined in. This means that if you import them into a file with a different version it won't change the way they're interpreted. Although, as we'll see later, the generated boostbook is slightly different.

As import now supports quickbook files, include also supports source files. It includes any quickbook contained in comments outside of code snippets. Code snippets in the file are available to be expanded within the file but are scoped to the file. In exactly the same manner as when templates and macros are scoped in an included quickbook file.

Id generation in quickbook 1.5 is a bit buggy, but that can't be fixed without a version switch as it will break existing documents. For example in quickbook 1.5 when you include a quickbook file, it stops using the explicit id from the documentation info and generates a new id from the document title to use instead.

The id generator in quickbook 1.6 has been improved in some other ways to. When generating ids from section titles, table titles etc. it always uses the quickbook source rather than the generated boostbook to generate the id. It then cleans up the id slightly, trimming leading and trailing underscores and replacing multiple underscores with a single underscore. Then if the newly generated part of the id is longer than 32 characters it truncates it.

While the new id generator generally creates better ids, it's more likely to generate duplicates so quickbook needs to handle duplicates better. When there are multiple identical ids, quickbook chooses one to use based on a priority list - anchors are preferred, then explicit document and section ids, then other explicit ids, followed by the generated ids. Then any other explicit ids in the document have numbers added to avoid duplicates - first the explicit ids in the order they appear and then the generated ids. A generated id which accidentally clashes with an explicit id should never change the explicit id.

Older language versions still generate the same ids they always have, with the exception of duplicate ids which are handled using the new mechanism - this is not a breaking change since duplicate ids can't be linked to.

As mentioned before, changing the id generator will break links in documents written using an old language version. So to ease the transition a 'compatibility mode' is used, this just requires an extra attribute in the docinfo, for example if you're converting a 1.5 document to 1.6:

[article Document
[quickbook 1.6]
[compatibility-mode 1.5]
]

This means the document will be parsed as 1.6, using all the new features, but ids (and possibly other markup) will generated as they were for a 1.5 document.

Compatibility mode is also implicitly used when generating templates written in a different language version to the current document. So the template is parsed in the version it was written for, but generates boostbook that's compatible with the current document.

Can now use quickbook and compatibility-mode tags at the beginning of the file. Either before or without a document info block. This is useful for files just containing templates, which don't really need a document info block.

If you don't specify compatibility-mode, the behaviour depends on whether or not you have a docinfo block. If you do it uses the file's quickbook version, if you don't it inherits the parent's compatibility mode even if you specify a quickbook version. This is the right thing to do - mixing compatibility modes within documents is problematic. It might actually be a mistake to allow them to specified outside docinfo blocks.

This change is also backdated to older versions. So when including from an older version, the included file's version can be set (older versions ignore document info in included files).

Headings can now be given explicit ids:

[heading:id A heading with an explicit id]

In 1.6, quickbook is more consistent about how it parses punctuation. Escapes are now supported in links, anchors, table titles, image attributes etc. The flip side of this is that quickbook is now stricter about unescaped brackets. They can still be used, but need to match up, otherwise there's an error.

Since quickbook now matches up square brackets it will fix some mis-parses. For example [*[bold]] used to parse as [bold] - note that the closing square bracket isn't bold, now it parses as [bold]. In this case it's just a subtle visual difference, but it could cause odd problems, for example when nested in a table cell.

Table titles are now parsed as phrases, so some markup is allowd:

[table [*bold title]]

Which is an empty table with a bold title. The title is no longer ended by a newline, but by either a closing square bracket, or two opening square brackets - which you get at the start of the table cells, so this now works:

[table Simple[[heading 1][heading 2]][[cell 1][cell 2]]]

A problem when using xi:include tags in escaped boostbook is that you typically don't know which directory the boostbook file will be in, so it's impossible to use relative links. This can be fixed by adding an xml:base attribute to the document tag. To do this use the new xmlbase attribute in your document's docinfo block. For example to make escaped xi:includes be relative to the directory of the file:

[library Library documentation
[quickbook 1.6]
[xmlbase .]
]

Any paths in xinclude elements will be rewritten accordingly. Note that most documents won't need this, and probably shouldn't use it. Only use it if you're totally sure that you will need it.

There's a new parser for template declarations and parameters which does a better job of understanding escaped and bracketed text. Unfortunately it does not understand element names so there are some cases where it could go wrong. For example:

[template doesnt_work[]
[ordered_list
    [`code phrase`]
]
]

In this case it will think the [\` is a template call and give a parse error. To work around this put an escaped space before the code phrase:

[template works[]
[ordered_list
    [\ `code phrase`]
]
]

New elements added in quickbook 1.6:

Previously if you used an element in the wrong context it would just be unprocessed, which was surprising. People often didn't realise that their element hadn't been processed. So now it's an error.

If the body of a phrase element didn't parse, it would be just used unprocessed. Now change it to be a hard error.

1.7 introduces a new ! element type for setting the source mode of a single entity without changing the source mode otherwise. This can be used for code blocks and other elements. For example:

[!c++]
    void foo() {};

[!python]```def foo():```

It can also be used to set the source mode for elements:

[!teletype][table
    [[code][meaning]]
    [[`+`][addition]]
]

When used before a section, it sets the source mode for the whole section.

If it appears at the beginning of a paragraph, it will be used for the whole paragraph only if there's a newline, eg.

[!c++]
A declaration `void foo();` and a definition `void foo() {}`.

Currently callouts can only be used in code snippets. 1.7 adds support in normal code blocks. The same syntax is used as in code snippets, the callout descriptions appear immediately after the code block.

Quickbook docinfo attributes will probably never be as rich as docbook attributes. To allow more flexible markup that is not supported by quickbook, escaped docbook can be included in the docinfo block:

[article Some article
[quickbook 1.7]
'''<author>
    <firstname>John</firstname>
    <surname>Doe</surname>
    <email>john.doe@example.com</email>
</author>'''
]

The escaped docbook is always placed at the end of the docinfo block, so it shouldn't be assumed that it will interleave with markup generated from quickbook. A mixture of quickbook and docbook attributes for the same information will not work well.

Paragraphs and block elements can now be used in lists:

* Para 1

  Para 2
  * Nested Para 1

    Nested Para 2

        Code block
  Para 3

generates:

There's support for calling templates in link values, anchors, roles and includes. This is sometimes a bit of a change, especially in places where spaces are currently allowed, so I might try using a slightly different grammar where required. I think I also need to add some validation, since the parser can allow more symbols than some of the old ones.

Can now place list markup in nested blocks, e.g in tables, variables lists etc. Unfortunately indented code blocks are more tricky, because the contents of these blocks are often indented already. It seemed easier to just not support indented code blocks in this context than to try to work out sensible actions for the edges cases. If you want to use code blocks in this context, you should still be able to use explicit markup.

Block elements can now be used in phrase templates, but paragraphs breaks aren't allowed, so this is an error:

[template paras[] Something or other.

Second paragraph.]

If a phrase template only contains block elements, then it's practically indistinguishable from a block template. So you'll get the same output from:

[template foo[] [blurb Blah, blah, blah]]

as:

[template foo[]
[blurb Blah, blah, blah]
]

If a phrase template has phrase content mixed with block elements, it'll generate output as if it was expanded inline.

One can now include multiple files at once using a glob pattern for the file reference:

[include sub/*/*.qbk]
[include include/*.h]

All the matching files, and intermediate irectories, will match and be included. The glob pattern can be "*" for matching zero or more characters, "?" for matching a single character, "[<c>-<c>]" to match a character class, "[^<char>-<char>]" to exclusive match a character class, "\\" to escape a glob special character which is then matched, and anything else is matched to the character.

	Note
	Because of the escaping in file references the "\\" glob escape is a double "\"; i.e. and escaped back-slash.