Making JavaScript Rulesets available to Picos


#1

Making JavaScript Rulesets available to Picos

If you look at the length of this post and think, “Too long,” in the tradition of Indigo Montoya, “Let me sum up:” to truly move Picos to a mainstream technology with an active community, it needs to be accessible in a mainstream language. That language, I believe, is JavaScript.

Background

Having worked with the Pico Labs projects for several months now, and being a forerunner of the layman in promoting Picos (I’ve only contributed a very small part to the code base), I feel it best to document my reasoning for suggesting that Pico rulesets should have the option of being written in JavaScript/Node.js.

The Problem

When a Pico is created for me on an engine owned by another person, I may be able to log into my Pico, but unless I know KRL, the Pico is useless to me. I have no way to give it new rules, nor do I have access to the engine to allow for modifications. Were Picos enabled to use JavaScript (Node.js) for ruleset development, it would allow myself and others to easily meet several advantages.

I will group them into the four following categories:

*Expedited adoption
*IoT interoperability
*Learning curve
*Native execution

Expedited Adoption

There was a recent event at Open West 2018 that was magnificent: each person at the conference could come to the Pico Labs booth and register for their own Pico, at which point they would then receive a printed QR code. When scanning the code of other participants (or being scanned by others), a mutual connection would be formed between the Picos of the respective owners. At the close of each day, whoever had the most connections won a prize.

The contest generated interest, the presentation was well-attended, yet none (to my knowledge) have expressed interest to assist in furthering the project. Participants were told that they owned their Pico, but what can they do with it? Without a knowledge of KRL, there is no purpose to log into the Pico in the first place.

I believe that the lack of JavaScript as a language choice to program the rules in hinders many people. I may receive a beautiful, leather-bound volume from a friend, but if it is written in any language besides English, it is of little use to me. Give me an English translation of Newton’s Principia and I’m all over it! (Please don’t; I already have a copy that I study. Get me a volume containing the collected works of James Clerk Maxwell instead, please.)

Node.js has a significant community, with numerous tools and IDE integrations. Every day people develop and make available new NPM modles; even the Pico Engine is distributed as an NPM module!! Should we make a module available that lets users write their own rules in JavaScript, it allows developers to quickly add functionality to Picos in a language that is familiar to them. It also enables them to develop their own modules for use with Picos that can be promoted on the Internet, vastly increase the adoption rate of Picos as a useful and viable technology.

IoT Interoperability

An individual can write rules to interface with IoT devices in KRL, but it takes time. Contrast this with the 769 NPM packages (at the time of this writing) that are designed to interface with a Raspberry Pi. One can almost drag-and-drop JavaScript code to control IoT systems: 139 packages under “temperature sensor”, 44 packages for “motion sensor”, 49 packages for “accelerometer”, and so forth. By allowing a user to leverage Node code for a ruleset, it enables quick development AND it promotes the community saying, “Hey, use a Pico for your IoT stuff; it’s easy!”

KRL Learning Curve

I know that this is a debatable subject, but when it comes to adoption, it’s a big one. Most people out there don’t even want to watch a YouTube video that’s over five minutes, let alone actually READ a post of this length! (Kudos to you for reading this far. Buy yourself a box of your favorite Kudos flavor at your local supermarket the next time that you’re out and imagine that I gave them to you.)

People want a quick answer and a solution that meets their needs (that’s why Stackoverflow is so popular; I can find the answer to my problem fast). For the general populace out there, “Learn KRL” is -not- a response that will resonate with them, especially when the documentation on Attlasian is pages and pages and pages long.

If, instead, we offer them JavaScript as a language that they can write rulesets with, MILLIONS of people only need to learn the APIs to be effective. JavaScript is a foundational language (at this point in time) that enterprise systems are built in, with, and around.

the Pico Engine runs on Node.js; it’s the underlying framework of the system. It is not a far cry to enable rules in the native language of the engine that executes them.

Native Execution

As mentioned before, the Pico Engine is already an NPM module. KRL is compiled down to run on Node.js, so why not just allow people to execute JavaScript rules directly? Instead of spending the parsing time, just plug-and-chug! Also, users can now leverage the abilities of Node.js to write complex rules with interactions to multiple devices.

Summary

A Pico that allows rules to be written in JavaScript is orders of magnitude more accessible to individuals, organizations, and communities than Picos are now. Picos allow for some excellent connections and ownership, but unless they can be utilized in a mainstream language, they will remain unused in common practical applications by the populace at large.

It is the suggestion of the post author that the Developers and Directors of the Pico Labs project give significant consideration to this proposal.


#2

I strongly agree that the Pico engine would benefit from supporting multiple languages and that the most natural language to support would be JavaScript. It must be noted that Picos are persistent compute object. In Object Oriented Programming it is common to create a base class with virtual methods that will be implemented at some point. Virtual methods require an understanding of the method’s intended use and restricted scopes. In comparison you can think of a Pico as a base class and KRL rules as virtual methods. Like virtual methods you will not be able to implement rules for a Pico until you understand what rules are intended for and what variable and permission scoping restrictions they have.
If the Pico engine supports other languages for developing rules, the steep learning curve of understanding what a rule is and does, will still persist. While supporting other languages for rules is most desirable, the engine would have to change some fundamental structural principals to meaningfully leverage any libraries in said, supported language.

The argument for other language support should come from dislikes or difficulties of KRL, for example.
KRL does not support…

a++;  
a = a + 1;

b = { gen_dynamic_key() : "a dynamic keys value"} // no dynamic keys in maps

for each item in array:    // no for loops
     ... // access item

if ( false ) { a=0;}
else {a=1;}

Instead you find yourself doing something like…

temp_var = a + 1;
a = temp_var; 

b = {}.put([gen_dynamic_key()], "a dynamic keys value")

array.map(function(item){ /* access item */ });

a = (false ) => 0 | 1;

Or you simply get upset with some small quark like having to use a semicolon on every line but the last line of an action block, while we are talking about action blocks lets ask, can you have a fired , not fired and always in the a same rule and of course, WHY NOT!!!

While the reasons to support more languages can still be added to, I will point out that the engine is still considered beta not having reached a version 1.00 release yet. So all the upsets of KRL are up for scrutiny and can be changed with a progressive form discussion and pull request.

I would like to see the Pico engine support JavaScript rules, I think the engine could accept a specific JavaScript rule class that is used to construct “rules” to be evaluated on a Pico. I do not know what this class would look like or how to declare and use entity variables in it or if this class is constructed and destructed for each event chain. But I would like to see it and use it.
I’m willing to help collaborate towards this goal.


#3

I believe that there could be some sort of API created that JavaScript can access. This would limit how the developers could create rules and provide more security. It would also abstract some of the difficulties in understanding the engine. “I need to raise an event, so I just call raisePicoEvent() with my data,” is what I’m going for. And we can make the API another NPM module so people just require that and they’re off and running!

I don’t intend for people to write rules that interact with the engine directly; for instance, to get data from a sensor, the ruleset could require an NPM module from the creator of the device and use it in their code, but then they can use the Pico API to interact with the engine as needed (i.e. raise an event when the temperature reaches a certain point or some such).

My limited understanding of the Pico engine may lead me to an oversimplification of the actual difficulties involved, but I believe that moving forward with JavaScript as a ruleset language is an important step in the advancement of Picos.


#4

I considered this in some detail back in 2013. There are a few things to consider:

  • Actor model - Picos are not object-oriented, they are actors. Thus there is a model disconnect between JavaScript and picos that has to be overcome.
  • Internet native - Picos are internet native. Node.js does some of that for Javascript (which is why the engine is written on top of Node.js).
  • Event-oriented - JavaScript does not natively understand events and doesn’t have support for event expressions, so you have to figure out how to select JavaScript objects using an event. This is more than just finding the right function to run, it’s finding all the right functions to run and running them. Then the results have to be combined in some coherent manner.
  • Persistence - Picos have persistent data elements that are segregated by ruleset and pico for entity variables and by ruleset only for application variables. This segregation isn’t arbitrary, it’s part of the Pico security model.
  • Modules - Picos have a module system that is not the same as JavaScripts object system. Again, there are security implications in how this is built so that a pico can safely run rulesets from multiple authors and use modules that are not explicitly permissioned in the pico.

I believe that some part of the pain in using KRL is the model, not the language. That is, if you build this into JavaScript, there will be many parts that are just as difficult and unfamiliar as they are in KRL without the language support to guide how they are used. So, your contention that “rules to be written in JavaScript is orders of magnitude more accessible to individuals” may or may not be true.

Here are some resources to look at in contemplating a design:

  • Pico Event Evaluation Cycle - This post looks at how events are handled in the pico engine at a conceptual level. Any JavaScript framework for programming picos would have to contend with this. Not impossible, but important.
  • A Programming Model for the CloudOS - Looks in detail about how to support JavaScript programming of Picos. Note that this post was written early in the development of Picos so the terminology is different than some of what we use today and some of the concepts are not fully developed. CloudOS == Wrangler, PDO == Pico

I would heartily support experimentation in this area. I believe the resulting system would need to support both JavaScript and KRL rulesets in the same pico.


#5

I am not advocating the removal of KRL in any way, nor do I believe that anyone thinks that I have implied such. I believe that the rulesets could be augmented by the addition of JavaScript.

The accessibility comes in the widespread knowledge of the JavaScript language, not in the use case of Picos. I will readily admit my ignorance with Picos on many levels. I merely want to suggest that this be investigated. There may be a limited API that a JavaScript developer could use to interact with Picos/the engine, or explicit restrictions could be imposed, e.g. “Promises will not work with Pico rulesets!” or whatever may be required.

It is often in the investigation of a new idea that other, more powerful, ideas are manifested. I hope to ignite that process. :grinning:


#6

Are you thinking of something like this ( https://github.com/hitch-b24/picosapi/wiki ), but for much lower level operations of the Pico-engine written in JavaScript?
The npm module would take a configuration that has a host that points to a Pico-engine and some sort of credential for that engine. Then from JavaScript you could write a program that uses a Pico or multiple Picos as the back end. I think this model is very doable and is the base structure of any UI for the Pico-engine and could/should be extended to be dynamic enough for this purpose.


#7

Yes! Something of that nature would be very useful for JavaScript.

We should also consider the other extensions mentioned earlier.


#8

Thinking about how

The ruleset is the unit of compilation for KRL and I believe ought to be the unit of interaction for JavaScript code that could run inside a pico. For background, the lifecycle of a ruleset is

  1. The ruleset is registered with a pico engine
  2. A registered ruleset is installed in a (pre-existing) pico
  3. The ruleset is available to respond to queries and participate in the pico’s reaction to events (not in this regard supplanting the reactions already occurring due to other rulesets in the pico)
  4. A ruleset might be uninstalled from a pico
  5. A ruleset might be unregistered from a pico engine

I can imagine a Node.js module providing/using hooks to handle the phase changes, and providing zero or more (JS) functions for responding to queries, and zero or more (JS) structures for participating in the reaction to events (as rules).

A JS function for responding to queries would be given an object allowing it read-only access to pico/ruleset entity variables, as well as the arguments coming in with the query. It would be expected to compute and return a JSON object (or just an Array, a number, a String, or null)

Each structure describing a rule would provide an object describing its selectivity (the incoming events to which it wishes to contribute a reaction) and a function which when called would be given an object containing all the event attributes, an object allowing it read/write access to pico/ruleset entity variables, and an object allowing it to contribute actions.

In phase 1 (register with engine) a node module would be created from the JS code which is compatible with the structure of node modules compiled from KRL (this will require some new work in the engine). After that, the new ruleset would participate fully with others installed in a pico.

Note that none of the KRL operators would be available in the JS code (but JS developers probably wouldn’t feel their lack). This outlined plan also doesn’t provide for JS rulesets to use/be KRL modules.


#9

Yes, this is important. The reaction to an event coming in to a pico is handled in a single thread, which is critical to the actor model from which picos inherit. Yet, JS has many ways (callbacks, promises, etc.) to circumvent this, and we’d have to be very careful to avoid all of them “for there are divers ways and means, even so many that I cannot number them.” Mosiah 4:29.


#10

More detail about how a

is given in a wiki page


#11

While there may be some benefit to KRL/JS module interchangeability, I agree that JS developers would probably never use a KRL module; they’d just create their own JavaScript version and push it out for others to use.

Without knowing the work required, I DO think that a Pico should be able to have KRL rulesets and JavaScript rulesets in it simultaneously, each coded completely in its respective language.


#12

I was thinking we could accomplish this with a web socket module. The module could have some sort of semi-fore with a timeout and provide access to entity variables in the rule its being called from. This module would pause a rules execution, notify NodeJs application over web socket about the rules selection, allow NodeJs application to alter any entity variables, and when the NodeJs application is done or a timeout has been reached, the rules execution would continue. I think this module would also provide a way to raise events over the web socket. A default ruleset for this module could exist so a NodeJs developer would only have to install a ruleset and never have to learn/use KRL. I think this approach would allow a KRL developer to integrate other programs into KRL as well as allow other programs to integrate with the pico-engine.
On the side of NodeJs, a npm library would need to be developed to connect to a pico over web socket and expose some sort of API for the web socket module as well as helper functions for sky-api.
I think it would look something like this.


#13

This turns out not to be true. Every pico has (currently) two rulesets installed in it, and there is a movement afoot to make that three. All of these rulesets (implemented in KRL, of course, at this point) provide functions and so can be used as modules by other rulesets. Some of these functions are incredibly valuable and in many cases there is no other way to get the information about the pico.

A JavaScript-developed ruleset would almost certainly need at least some of that information, and so it must be possible for JS rulesets to use KRL rulesets as modules. A brief analysis of modules “written in” and “used from” the two languages:

Obviously we can now write a ruleset in KRL and use it as a module in another ruleset also written in KRL. By the argument above, we need to be able to use such modules in a ruleset written in JS.