native OVOS integration

[JarbasAl]

hello! just found out about this project thanks to @PureTryOut

i am not familiar with GTK or with this code base, i took a quick look at the repo. apologies if my terminology is off or i misunderstood something

my suggestion is that this is made to support all the SYSTEM_XXX.qml equivalents, these correspond to templates commonly used by skills, eg, self.gui.show_image, self.gui.show_text, docs here and can be easily parsed from the requested page name

regarding the hardcoding of skill_ids to views, i wonder how we could best allow skills to provide this (optionally)

gui_wrappers = {
    "mycroft-fallback-duck-duck-go.mycroftai": AnswerImageWrapper,
    "mycroft-wiki.mycroftai": AnswerImageWrapper
}

we have been discussing the best way to support other GUIs that don't use QML such as javascript or command line interfaces, the easy implementation would be to check if the skill provides a equivalent named file with different file extensions, ie

• .qml
• .ui
• .html
• .jsx

if we implement this in ovos, would the .ui format seen here be the way to go from skill developers side if they wanted to explicitly support lapel or other GTK based gui clients?

[knuxify]

It's been a while since I properly looked into the lapel source code, so my memory's a bit rusty, but I'll try my best to answer any questions. (Would be nice to get this project up and running again...)

if we implement this in ovos, would the .ui format seen here be the way to go from skill developers side if they wanted to explicitly support lapel or other GTK based gui clients?

I'm not entirely sure whether something like the QML approach could be done with GTK's .ui files. In general, the way UI files work is a bit confusing:

• There's the raw Gtk.Builder machinery, but that's mostly for constructing entire user interfaces, and has mostly been phased out in favor of templates:
• Templates, loaded through the template machinery (gtk_widget_set_template, gtk_widget_init_template - though these functions are abstracted in most language bindings, e.g. in PyGObject (which this project uses), there's a Gtk.Template() decorator that can be used on a class definition)).

The main problem is that templates have to be bound to a widget class, so we'd effectively have to add some way for the code to make a class on-the-fly for those UI files to use. Perhaps I'm understanding this wrong - I'm not nearly deep enough in GTK internals to figure out whether, and how, this would be possible. There's probably a way to do this, but it sounds like a lot of work. (What I mean is, we could probably do this for the system QMLs and just add a submodule to OVOS that has PyGObject-compatible objects, but skills adding their own UI files would have to use this hacky mechanism I described.)

In general, though, I find the approach used by Mycroft with QML files a bit awkward - if I understand correctly, they are made to work primarily in a full-screen setting, like the standard Mycroft-GUI. Fitting them into an app like Lapel, where content is provided in a chat-like format akin to Google Assistant, could be challenging.

An alternative could be providing the data needed to build an interface in a more easily parsable format, like JSON. Then we could e.g. add "display types" which would serve effectively the same purpose as the SYSTEM qml files (I'm thinking kinda like the card types of Twitter Cards), "actions" which would denote things like buttons, and maybe some other options for more specific layouts. This way there's an unified way to specify what the UI should contain, and GUI makers could choose how they want to display the information. It also means it can be used in a lot more than just QML and GTK apps, without skill devs having to support each one separately.

...still, that also sounds like plenty of development effort, but it might be a bit more sustainable. This is all just a loose suggestion though :p

I'd love to discuss this further, I'm open to suggestions!

[knuxify]

...actually, disregard everything I said about templates, I think we should be able to do the same thing as the QML files with GtkBuilder. See gtk_builder_get_object() - this just gets a GTK widget from the template.