Use a plugin system to refactor the life cycle of a PyScript page

TL;DR: I'm proposing to model the life cycle of a PyScript page using a plugin system. The main logic is very slim, it calls hooks at well defined times, and most of the functionality is implemented inside plugins, some of which are builtin. This is loosely modeled after what e.g. pytest does, but I'm sure there are other systems following a similar model

In order to design the new system, we need to first analyze what is the current flow of execution. To avoid having a giant wall of text, I'm going to analyze the current status in this post, and write a proposal for the refactoring in the next one.

Current control flow

The HTML page is parsed. We encounter <script defer src=".../pyscript.js> Because of defer, the script is executed when the document is fully parsed.

pyscript.js is executed. We start from main.ts:

pyscript/pyscriptjs/src/main.ts

Lines 1 to 43 in 6023c41

    
           import './styles/pyscript_base.css'; 
        
           import { PyScript } from './components/pyscript'; 
        
           import { PyRepl } from './components/pyrepl'; 
        
           import { PyEnv } from './components/pyenv'; 
        
           import { PyBox } from './components/pybox'; 
        
           import { PyButton } from './components/pybutton'; 
        
           import { PyTitle } from './components/pytitle'; 
        
           import { PyInputBox } from './components/pyinputbox'; 
        
           import { PyWidget } from './components/base'; 
        
           import { PyLoader } from './components/pyloader'; 
        
           import { globalLoader } from './stores'; 
        
           import { PyConfig } from './components/pyconfig'; 
        
           import { getLogger } from './logger'; 
        
           const logger = getLogger('pyscript/main'); 
        
           /* eslint-disable @typescript-eslint/no-unused-vars */ 
        
           const xPyScript = customElements.define('py-script', PyScript); 
        
           const xPyRepl = customElements.define('py-repl', PyRepl); 
        
           const xPyEnv = customElements.define('py-env', PyEnv); 
        
           const xPyBox = customElements.define('py-box', PyBox); 
        
           const xPyButton = customElements.define('py-button', PyButton); 
        
           const xPyTitle = customElements.define('py-title', PyTitle); 
        
           const xPyInputBox = customElements.define('py-inputbox', PyInputBox); 
        
           const xPyWidget = customElements.define('py-register-widget', PyWidget); 
        
           const xPyLoader = customElements.define('py-loader', PyLoader); 
        
           const xPyConfig = customElements.define('py-config', PyConfig); 
        
           /* eslint-enable @typescript-eslint/no-unused-vars */ 
        
           // As first thing, loop for application configs 
        
           logger.info('checking for py-confing'); 
        
           const config: PyConfig = document.querySelector('py-config'); 
        
           if (!config) { 
        
               const loader = document.createElement('py-config'); 
        
               document.body.append(loader); 
        
           } 
        
           // add loader to the page body 
        
           logger.info('add py-loader'); 
        
           const loader = <PyLoader>document.createElement('py-loader'); 
        
           document.body.append(loader); 
        
           globalLoader.set(loader);

main.ts imports a bunch of modules; in particular, pyscript.ts has also some module-level statements which call addInitializer and addPostInitializer:

pyscript/pyscriptjs/src/components/pyscript.ts

Lines 265 to 266 in 6023c41

addInitializer(mountElements);

addPostInitializer(initHandlers);

we call customElements.define for all CE. As soon as each of them is defined, all the tags in the document are connected. This means that the order of execution of the callbacks might be different than the order of the tags in the HTML. Related issue: <py-script>, <py-button> and <py-inputbox> etc. are executed in a non-obvious order #761.
Note: that at this point in time nothing else is loaded yet; no runtime, no py-config, no py-env, etc.
Scripts are enqueued but not executed.

pyscript/pyscriptjs/src/main.ts

Lines 19 to 28 in 6023c41

    
           const xPyScript = customElements.define('py-script', PyScript); 
        
           const xPyRepl = customElements.define('py-repl', PyRepl); 
        
           const xPyEnv = customElements.define('py-env', PyEnv); 
        
           const xPyBox = customElements.define('py-box', PyBox); 
        
           const xPyButton = customElements.define('py-button', PyButton); 
        
           const xPyTitle = customElements.define('py-title', PyTitle); 
        
           const xPyInputBox = customElements.define('py-inputbox', PyInputBox); 
        
           const xPyWidget = customElements.define('py-register-widget', PyWidget); 
        
           const xPyLoader = customElements.define('py-loader', PyLoader); 
        
           const xPyConfig = customElements.define('py-config', PyConfig);

The various connectedCallback are called. In the following, we say "DOM logic" to mean all the logic which aims at locating/adding/modifying elements in the DOM, e.g. to create the <div> to hold the output of a script:
- PyScript: some immediate DOM logic, then it calls addToScriptsQueue
- PyRepl: only immadiate DOM logic
- PyEnv: very few logic done immediately; it uses addInitializer to call loadEnv and loadPaths later
- PyBox, PyTitle: only immadiate DOM logic
- PyButton, PyInputBox: immediate DOM logic + this.runAfterRuntimeInitialized (which uses runtimeLoaded.subscribe)
- PyWidget: after a bit of indirections, it ends up calling
  runtimeLoaded.subscribe, but then in the callback we use a 1sec delay
  before actually instantiating proxyClass (which is a python class)
- PyConfig: see later
after the imports, there is some core logic in main.ts: we create a <py-config> element if needed. So precise timing at which PyConfig.connectedCallback happens depends on whether there is a real tag or not:
- if there was a <py-config> tag, it was called at point (4)
- else, it is called now.

pyconfig.connectedCallback calls loadRuntimes(), which create a <script> tag to download pyodide, and add an event listener for its load() event:

pyscript/pyscriptjs/src/components/pyconfig.ts

Lines 62 to 73 in 6023c41

    
           loadRuntimes() { 
        
               logger.info('Initializing runtimes'); 
        
               for (const runtime of this.values.runtimes) { 
        
                   const runtimeObj: Runtime = new PyodideRuntime(runtime.src, runtime.name, runtime.lang); 
        
                   const script = document.createElement('script'); // create a script DOM node 
        
                   script.src = runtimeObj.src; // set its src to the provided URL 
        
                   script.addEventListener('load', () => { 
        
                       void runtimeObj.initialize(); 
        
                   }); 
        
                   document.head.appendChild(script); 
        
               } 
        
           }

we continue the main flow of execution: in main.ts we add the <py-loader> element to the DOM; the splashscreen is showed only now. main.ts finally finishes execution:

pyscript/pyscriptjs/src/main.ts

Lines 39 to 43 in 6023c41

    
           // add loader to the page body 
        
           logger.info('add py-loader'); 
        
           const loader = <PyLoader>document.createElement('py-loader'); 
        
           document.body.append(loader); 
        
           globalLoader.set(loader);

eventually the <script> tag added at point (7) finishes loading, and it fires the load event (which is implemented inside pyconfig.ts:loadRuntimes). We call runtimeObj.initialize(). The execution flow bounces back and forth between the base class in runtime.ts and the concrete subclass in pyodide.ts. We call loadInterpreter.

pyscript/pyscriptjs/src/runtime.ts

Lines 135 to 181 in 6023c41

    
           async initialize(): Promise<void> { 
        
               loader?.log('Loading runtime...'); 
        
               await this.loadInterpreter(); 
        
               const newEnv = { 
        
                   id: 'default', 
        
                   runtime: this, 
        
                   state: 'loading', 
        
               }; 
        
               runtimeLoaded.set(this); 
        
               // Inject the loader into the runtime namespace 
        
               // eslint-disable-next-line 
        
               this.globals.set('pyscript_loader', loader); 
        
               loader?.log('Runtime created...'); 
        
               loadedEnvironments.update(environments => ({ 
        
                   ...environments, 
        
                   [newEnv['id']]: newEnv, 
        
               })); 
        
               // now we call all initializers before we actually executed all page scripts 
        
               loader?.log('Initializing components...'); 
        
               for (const initializer of initializers_) { 
        
                   await initializer(); 
        
               } 
        
               loader?.log('Initializing scripts...'); 
        
               for (const script of scriptsQueue_) { 
        
                   await script.evaluate(); 
        
               } 
        
               scriptsQueue.set([]); 
        
               // now we call all post initializers AFTER we actually executed all page scripts 
        
               loader?.log('Running post initializers...'); 
        
               if (appConfig_ && appConfig_.autoclose_loader) { 
        
                   loader?.close(); 
        
               } 
        
               for (const initializer of postInitializers_) { 
        
                   await initializer(); 
        
               } 
        
               // NOTE: this message is used by integration tests to know that 
        
               // pyscript initialization has complete. If you change it, you need to 
        
               // change it also in tests/integration/support.py 
        
               logger.info('PyScript page fully initialized'); 
        
           }

We call all the things which were enqueued earlier. We start with initializers, which currently are:

pyscript.ts:mountElements:

pyscript/pyscriptjs/src/components/pyscript.ts

Lines 254 to 265 in 6023c41

    
           async function mountElements() { 
        
               const matches: NodeListOf<HTMLElement> = document.querySelectorAll('[py-mount]'); 
        
               logger.info(`py-mount: found ${matches.length} elements`); 
        
               let source = ''; 
        
               for (const el of matches) { 
        
                   const mountName = el.getAttribute('py-mount') || el.id.split('-').join('_'); 
        
                   source += `\n${mountName} = Element("${el.id}")`; 
        
               } 
        
               await runtime.run(source); 
        
           } 
        
           addInitializer(mountElements);

pyenv.ts:loadEnv and loadPaths:

pyscript/pyscriptjs/src/components/pyenv.ts

Lines 59 to 79 in 6023c41

    
           async function loadEnv() { 
        
               logger.info("Loading env: ", env); 
        
               await runtime.installPackage(env); 
        
           } 
        
           async function loadPaths() { 
        
               logger.info("Paths to load: ", paths) 
        
               for (const singleFile of paths) { 
        
                   logger.info(`  loading path: ${singleFile}`); 
        
                   try { 
        
                       await runtime.loadFromFile(singleFile); 
        
                   } catch (e) { 
        
                       //Should we still export full error contents to console? 
        
                       handleFetchError(<Error>e, singleFile); 
        
                   } 
        
               } 
        
               logger.info("All paths loaded"); 
        
           } 
        
           addInitializer(loadEnv); 
        
           addInitializer(loadPaths);

we execute <py-script> tags:

pyscript/pyscriptjs/src/runtime.ts

Lines 161 to 165 in 6023c41

    
           loader?.log('Initializing scripts...'); 
        
           for (const script of scriptsQueue_) { 
        
               await script.evaluate(); 
        
           } 
        
           scriptsQueue.set([]);

We close the loader (why so early?):

pyscript/pyscriptjs/src/runtime.ts

Lines 170 to 172 in 6023c41

    
           if (appConfig_ && appConfig_.autoclose_loader) { 
        
               loader?.close(); 
        
           }

we run postInitializers. Currently the only one is initHandlers:

pyscript/pyscriptjs/src/components/pyscript.ts

Lines 213 to 219 in 6023c41

    
           /** Initialize all elements with py-* handlers attributes  */ 
        
           async function initHandlers() { 
        
               logger.debug('Initializing py-* event handlers...'); 
        
               for (const pyAttribute of pyAttributeToEvent.keys()) { 
        
                   await createElementsWithEventListeners(runtime, pyAttribute); 
        
               } 
        
           }

at some point, we also run the callbacks registered by using runAfterRuntimeInitialized, which currently is invoked by:

pybutton.ts
pyinputbox.ts

in both cases above, what it does is to run further user-define code inside e.g. the <py-button> tag, but using a different way than for <py-script>:

pyscript/pyscriptjs/src/components/pybutton.ts

Lines 69 to 73 in 6023c41

    
           this.runAfterRuntimeInitialized(async () => { 
        
               await this.eval(this.code); 
        
               await this.eval(registrationCode); 
        
               logger.debug('registered handlers'); 
        
           });

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	import './styles/pyscript_base.css';

	import { PyScript } from './components/pyscript';
	import { PyRepl } from './components/pyrepl';
	import { PyEnv } from './components/pyenv';
	import { PyBox } from './components/pybox';
	import { PyButton } from './components/pybutton';
	import { PyTitle } from './components/pytitle';
	import { PyInputBox } from './components/pyinputbox';
	import { PyWidget } from './components/base';
	import { PyLoader } from './components/pyloader';
	import { globalLoader } from './stores';
	import { PyConfig } from './components/pyconfig';
	import { getLogger } from './logger';

	const logger = getLogger('pyscript/main');

	/* eslint-disable @typescript-eslint/no-unused-vars */
	const xPyScript = customElements.define('py-script', PyScript);
	const xPyRepl = customElements.define('py-repl', PyRepl);
	const xPyEnv = customElements.define('py-env', PyEnv);
	const xPyBox = customElements.define('py-box', PyBox);
	const xPyButton = customElements.define('py-button', PyButton);
	const xPyTitle = customElements.define('py-title', PyTitle);
	const xPyInputBox = customElements.define('py-inputbox', PyInputBox);
	const xPyWidget = customElements.define('py-register-widget', PyWidget);
	const xPyLoader = customElements.define('py-loader', PyLoader);
	const xPyConfig = customElements.define('py-config', PyConfig);
	/* eslint-enable @typescript-eslint/no-unused-vars */

	// As first thing, loop for application configs
	logger.info('checking for py-confing');
	const config: PyConfig = document.querySelector('py-config');
	if (!config) {
	const loader = document.createElement('py-config');
	document.body.append(loader);
	}

	// add loader to the page body
	logger.info('add py-loader');
	const loader = <PyLoader>document.createElement('py-loader');
	document.body.append(loader);
	globalLoader.set(loader);

	addInitializer(mountElements);
	addPostInitializer(initHandlers);

	loadRuntimes() {
	logger.info('Initializing runtimes');
	for (const runtime of this.values.runtimes) {
	const runtimeObj: Runtime = new PyodideRuntime(runtime.src, runtime.name, runtime.lang);
	const script = document.createElement('script'); // create a script DOM node
	script.src = runtimeObj.src; // set its src to the provided URL
	script.addEventListener('load', () => {
	void runtimeObj.initialize();
	});
	document.head.appendChild(script);
	}
	}

	async initialize(): Promise<void> {
	loader?.log('Loading runtime...');
	await this.loadInterpreter();
	const newEnv = {
	id: 'default',
	runtime: this,
	state: 'loading',
	};
	runtimeLoaded.set(this);

	// Inject the loader into the runtime namespace
	// eslint-disable-next-line
	this.globals.set('pyscript_loader', loader);

	loader?.log('Runtime created...');
	loadedEnvironments.update(environments => ({
	...environments,
	[newEnv['id']]: newEnv,
	}));

	// now we call all initializers before we actually executed all page scripts
	loader?.log('Initializing components...');
	for (const initializer of initializers_) {
	await initializer();
	}

	loader?.log('Initializing scripts...');
	for (const script of scriptsQueue_) {
	await script.evaluate();
	}
	scriptsQueue.set([]);

	// now we call all post initializers AFTER we actually executed all page scripts
	loader?.log('Running post initializers...');

	if (appConfig_ && appConfig_.autoclose_loader) {
	loader?.close();
	}

	for (const initializer of postInitializers_) {
	await initializer();
	}
	// NOTE: this message is used by integration tests to know that
	// pyscript initialization has complete. If you change it, you need to
	// change it also in tests/integration/support.py
	logger.info('PyScript page fully initialized');
	}

	async function mountElements() {
	const matches: NodeListOf<HTMLElement> = document.querySelectorAll('[py-mount]');
	logger.info(`py-mount: found ${matches.length} elements`);

	let source = '';
	for (const el of matches) {
	const mountName = el.getAttribute('py-mount') \|\| el.id.split('-').join('_');
	source += `\n${mountName} = Element("${el.id}")`;
	}
	await runtime.run(source);
	}
	addInitializer(mountElements);

	async function loadEnv() {
	logger.info("Loading env: ", env);
	await runtime.installPackage(env);
	}

	async function loadPaths() {
	logger.info("Paths to load: ", paths)
	for (const singleFile of paths) {
	logger.info(` loading path: ${singleFile}`);
	try {
	await runtime.loadFromFile(singleFile);
	} catch (e) {
	//Should we still export full error contents to console?
	handleFetchError(<Error>e, singleFile);
	}
	}
	logger.info("All paths loaded");
	}

	addInitializer(loadEnv);
	addInitializer(loadPaths);

	/** Initialize all elements with py-* handlers attributes */
	async function initHandlers() {
	logger.debug('Initializing py-* event handlers...');
	for (const pyAttribute of pyAttributeToEvent.keys()) {
	await createElementsWithEventListeners(runtime, pyAttribute);
	}
	}

	this.runAfterRuntimeInitialized(async () => {
	await this.eval(this.code);
	await this.eval(registrationCode);
	logger.debug('registered handlers');
	});

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions