All about the development of Exponential (CMS) by Graham Brookins, 7x (formerly Brookins Consulting) and our think tank kracker.org.
Hello Exponential Community,
7x keeps the complete doc.ez.no documentation as a static HTML archive in a public GitHub repository. Two free Exponential extensions turn that archive, or any folder of HTML or Markdown, or a live website, into real content objects in your own Exponential site with one command. One run gives finished work; running it again is never needed, but always safe. The result runs live at doc.exponential.earth.
The repositories
- se7enxweb/exponential: Exponential 6.0.15 or later, the CMS.
- se7enxweb/exponential.doc.exponential.earth: the static archive of doc.ez.no, 88,680 pages.
- se7enxweb/exp_static_html_imports: the import framework. Four source types, your own types, safe re-runs, reports.
- se7enxweb/exp_doc: documentation content classes, the import rules for the archive, and two designs: Classic (doc2025, the static site exactly) and Modern (doc2026).
All are free software under the GNU General Public License v2.0 (or any later version).
What you need
- Exponential 6.0.15 or later (earlier 6.0 versions work without the audit events and the console aliases).
- PHP 7.4 or later; PHP 8.0 to 8.5 are supported.
- Any of five databases: MySQL/MariaDB, PostgreSQL, SQLite, Oracle or MongoDB.
- The online editor ezoe, active by default. Its HTML parser turns pages into rich text.
- git on the server, for sources given as a repository.
- For exp_doc also: eztags (labels) and xrowmetadata (the metadata attribute).
Example 1: a folder of Markdown, in minutes
Install the framework from your Exponential root:
git clone https://github.com/se7enxweb/exp_static_html_imports.git extension/exp_static_html_imports
Activate it in settings/override/site.ini.append.php:
[ExtensionSettings] ActiveExtensions[]=exp_static_html_imports
Deploy, check, look first, then import:
php bin/php/ezpgenerateautoloads.php -e php bin/php/ezcache.php --clear-all php bin/php/console exp:static-import:types php bin/php/console exp:static-import:run --markdown --source=/path/to/docs --parent-node=2 --dry-run php bin/php/console exp:static-import:run --markdown --source=/path/to/docs --parent-node=2 php bin/php/console exp:static-import:status
- exp:static-import:types lists the four types: --doc-html, --html-generic, --markdown, --sitemap-crawl.
- --dry-run counts what would be created, updated, flagged and removed, writes a report, and changes nothing.
- Each Markdown file (.md, .markdown) becomes a Folder below node 2: name from the title, short description from the body. README.md and index.md are the index page of their folder. Front matter is read; code blocks keep their language.
- On Velocity, php bin/php/console exp:velocity deploy --allow-root-user replaces the two deploy lines.
Example 2: a folder of HTML, or a live website
php bin/php/console exp:static-import:run --html-generic --source=/path/to/html --parent-node=2 --dry-run php bin/php/console exp:static-import:run --html-generic --source=/path/to/html --parent-node=2 php bin/php/console exp:static-import:run --sitemap-crawl --source=https://www.example.com --parent-node=2
- --html-generic reads .html and .htm files; index.html is the folder page. The title comes from h1, the body from main, article or body; nav, header and footer are removed; code in pre keeps its language. Linked .pdf and .zip files become File objects, other assets Image objects.
- --sitemap-crawl reads the pages in the site's sitemap.xml politely: robots.txt respected, one request a second, its own user agent, and each page fetched once.
- --path='docs/*' imports only the matching paths (shell patterns) and --limit=50 only 50 pages: a quick first look at a large source.
Example 3: a real site, imported in 37 seconds: Learn X in Y minutes
doc.exponential.earth/learninminutes is Learn X in Y minutes (CC BY-SA 3.0): 201 programming language tours, imported straight from GitHub into a siteaccess with its own content tree. This is everything it took.
1. The import type, in settings/override/statichtmlimport.ini.append.php:
[ImportTypes] TypeList[learnx]=expStaticImportTypeLearnX [Type_learnx] SourceRepository=https://github.com/adambard/learnxinyminutes-docs.git ParentRemoteID=learninminutes_site_root Suffixes[] Suffixes[]=md IncludePaths[] IncludePaths[]=*.md ExcludePaths[] ExcludePaths[]=README.md ExcludePaths[]=CONTRIBUTING.md DefaultClass=exp_doc_page AttributeMap[] AttributeMap[exp_doc_page/title]=title AttributeMap[exp_doc_page/body]=body AttributeMap[exp_doc_page/original_html]=original_html AttributeMap[exp_doc_page/source_path]=source_path
- SourceRepository: the import clones the repository itself and pulls it on every later run.
- IncludePaths[]=*.md takes the English pages at the top; the translations live in folders (de/, fr/ ...), and a pattern like de/*.md imports one of them.
- exp_doc_page is a class of exp_doc; any class with a title and a rich text body works (article/title, article/body).
2. The one complication, and its easy fix. These pages have no title in their front matter, and their first "# " line is a code comment, so the dry run named the Python page "Single line comments start with a number symbol." A type of your own, in an extension of your own, names each page by the front matter's name instead. Save it as extension/learninminutes/classes/expstaticimporttypelearnx.php, add ActiveExtensions[]=learninminutes to settings/override/site.ini.append.php, and run php bin/php/ezpgenerateautoloads.php -e:
<?php
class expStaticImportTypeLearnX extends expStaticImportTypeMarkdown
{
public function identifier()
{
return 'learnx';
}
public function parse( $source, $path )
{
$page = parent::parse( $source, $path );
list( $front ) = self::splitFrontMatter( $page['original_html'] );
$meta = self::frontMatter( $front );
$name = isset( $meta['name'] ) ? $meta['name'] : ( isset( $meta['language'] ) ? $meta['language'] : '' );
if ( is_string( $name ) && trim( $name ) !== '' )
$page['title'] = trim( $name );
return $page;
}
}3. A siteaccess with its own content tree root:
- A section page (or folder) for the site's root, below the Websites node, with the remote id learninminutes_site_root.
- settings/siteaccess/learninminutes/: a copy of your front siteaccess with content.ini [NodeSettings] RootNode, site.ini IndexPage and DefaultPage (/content/view/full/<root node>), PathPrefix (the root's URL path), SiteURL and SiteName set for the new site.
- learninminutes added to SiteList[], AvailableSiteAccessList[] and RelatedSiteAccessList[] in settings/override/site.ini.append.php.
- The Anonymous role's user/login policy allowed on the new siteaccess, or visitors are refused.
- Caches cleared: php bin/php/ezcache.php --clear-tag=ini, and with DesignLocationCache=enabled also --clear-id=design_base,template,template-override.
4. Look first, then import:
php bin/php/console exp:static-import:run --learnx --dry-run php bin/php/console exp:static-import:run --learnx
- Dry run: 201 pages, nothing written, 6.1 seconds.
- Import: 201 pages created, 0 failed, 36.7 seconds.
- Again, nothing changed upstream: 201 unchanged, 5.7 seconds.
- Again, after a rendering fix: only the 141 changed pages updated, 60 unchanged, 21.6 seconds. Nothing to clean up by hand.
Use exp_static_html_imports from 2026-11-07 or later: before it, Markdown pages on Exponential 6.0.15 were stored as the word "Array", and wrapped lines lost their spaces. This import found both; the fix is commit e4a240c.
Your own classes and sources
Copy the type's group from extension/exp_static_html_imports/settings/statichtmlimport.ini.append.php into settings/override/statichtmlimport.ini.append.php and change it. For example, HTML pages as Articles:
[Type_html-generic] SourceLocation=/path/to/html ParentNode=2 DefaultClass=article AttributeMap[] AttributeMap[article/title]=title AttributeMap[article/body]=body
- Source: SourceLocation (a folder) or SourceRepository (a git URL the import clones and pulls).
- Place: ParentNode, or ParentRemoteID for a node with that remote id.
- Classes: DefaultClass, ClassRules[] (a class per path pattern) and AttributeMap[] (class/attribute=field).
- Rules are INI by default, or YAML, or a PHP rule class (RulesFormat).
- A source type of your own is a PHP class implementing expStaticImportTypeInterface, registered in statichtmlimport.ini [ImportTypes] TypeList[].
- Every rule is described in the extension's doc/rules.md, also shown in the help center when the extension is active.
Example 4: your own copy of the doc.ez.no archive
git clone https://github.com/se7enxweb/exp_static_html_imports.git extension/exp_static_html_imports git clone https://github.com/se7enxweb/exp_doc.git extension/exp_doc
Activate both in settings/override/site.ini.append.php, in this order:
[ExtensionSettings] ActiveExtensions[]=exp_static_html_imports ActiveExtensions[]=exp_doc
Give the folder that will be the documentation site's root the remote id exp_doc_site_root, then:
php bin/php/ezpgenerateautoloads.php -e php bin/php/ezcache.php --clear-all php bin/php/console exp:migration:run --extension=exp_doc php bin/php/console exp:doc:import --path='FAQ/*' --dry-run php bin/php/console exp:doc:import --path='FAQ/*' php bin/php/console exp:doc:import
- The migrations create the class group Documentation with its classes (pages, sections, manuals, versions, tutorials and their steps, FAQ groups and their questions, extension reference pages, images and files, each with a Bootstrap icon and a metadata attribute), the sections Doc, Media and Hidden, the role Doc reader for anonymous read access, and the media folder.
- exp:doc:import clones the archive from its repository by itself and imports it with the documentation's rules (exp_doc/settings/statichtmlimport.ini.append.php). It takes the same options as exp:static-import:run.
- --path='FAQ/*' imports one part in minutes. The whole archive is 88,680 pages: its final run took 69 minutes on our server. An interrupted run continues where it stopped when you run the same command again.
- Design: set SiteDesign=doc (Modern) or SiteDesign=doc2025 (Classic) in the front siteaccess's site.ini.append.php. The header switches between them with one click.
- Cronjobs: run runcronjobs.php frequent every five minutes. It reads the front page's commit and forum feeds and recounts the Contributors page; without it both still work, more slowly.
- SQLite: after a large import run PRAGMA analysis_limit = 1000; ANALYZE; again (a migration does it once). It changes no data and keeps content fetches fast.
- Duplicate contributors: php bin/php/console exp:doc:contributor-merge --into=<contributor> --from=<duplicate> [--dry-run].
One run is enough; running again is safe
- Identity: every page is the object with remote id staticimport-<type>-<md5 of its source path>, wherever it has been moved.
- Changed source: a new version is published. Unchanged: nothing is written.
- Edited in the CMS: the page is flagged and left alone; --overwrite-flagged writes it anyway and the edit is lost.
- Gone from the source: the node is hidden and reported; --apply-removals moves it to the trash.
- Copies of a page and redirect stubs become redirects to it; error pages are skipped. Every old path, with and without .html, redirects to its new node.
- Large sources run in batches (--batch, 50 by default) and save their position after each; the same command resumes, --restart starts over, --no-pull skips the git pull.
- Every run writes a JSON report (schema exp-static-import/1), a Markdown summary and audit trail events.
- In the administration, Setup > Static site import shows the runs, starts one as a background job, and opens each report with one click.
Questions, ideas and pull requests are welcome on GitHub. Enjoy!
I am very excited as we have officially confirmed, after much other work recently (more coming soon) that our Original Support request to the GitHub CLI Team to support our story to enable users to modify via github cli script repository sponsorship meta data changes to enable or disable repository sponsorship button / meta data display if a .github/FUNDING.yml file is present and configured.
This means with the documentation provided by the GitHub CLI Support Team you too can have thousands of repositories and enable sponsorship information dynamically for all of them! With a little work, it can be done faster with our support!
This quick simple supported solution is proudly offered here to others with a similar need. We hope this helps you reach your own goals faster!
We were able to enable sponsorship buttons on over 884 repositories in less than 2 hours all without manual clicking via simple scripting.
Looking for a WordPress alternative that is a compelling case for better organized software platforms in general? Try Exponential it will certainly open up your eyes to continued possibilities not dead ends. Give Exponential a try today!
Enjoy Everyone!
Cheers!
This is just too cool for words. It’s really coming true, SQLite Database Support equal to MySQL or Oracle but drastically cheaper in cost to get started quickly.
Have you ever wished to reduce your own database headaches with eZ Publish?
For years I’ve watched others developing with eZ and wished silently for the day when SQLite database driver support in the eZ Publish kernel and setup wizard would be generally available to end users.
I have grown from not just dreaming but stepping up, reaching my goals and making my dreams come true with hard work.
Checkout the newly refreshed installation process. We lightly refactored the default setup wizard to provide for a completely simple database selection choice and full installation support for site package data and site configuration used when building a SQLite driven database eZ Publish website, kiosk app or other unique use cases.
With this new feature of our software eZ Publish 6.0.1 (github/main;unreleased) you on need to install PHP, PHP Extensions (required by eZ Publish which now includes the SQLite driver which is painless to install using your favorite OS package manager). As a Debian advocate I recommend APT which is a time saver in life in all things. Back to the point after php is installed you only need to install composer! That’s it! Run the php built in web server and install a full working eZ Publish installation in minutes instead of hours or potentially days. This is a big deal for people who just want to use the software without having the requirements of a full web server or database server setup just to try using the latest version.
The quickest way to get started is to install via composer using the main branch of the project version control.
Here is to a new year full of exciting developments in eZ Publish History.
Today 7x was elevated on GitHub as a sponserable account.
This means that today onward our supporters can easily choose the sponsor level that fits their own needs and financially support the human powered work that drives eZ Publish forward to greater features and maintenance.
It seems we are the first eZ Publish developers to setup flexible financial support options via, GitHub Sponsors, Patreon and PayPal. This is a good sign as we want to be the change we want in life and work.
Sponsor 7x today at a level that your comfortable with and help us in a big way.
Add comment