DXP Framework

Tips & Tricks

Setting Up A Developer Environment

To efficiently debug while running the experience do the following:

  • In the DXP installation folder (typically "C:\PRSONAS"). In the "Viewer" folder open the file PRSONAS_VIEWER.exe.config.
  • Locate the key RemoteDebugPort and set the value to 7373
  • Locate the key WebServerRootFolder and set the value to the path of your development folder. This will force the experience to run the UI from this folder.
  • Open up the Chrome inspector to this URL chrome://inspect/#devices.

When the experience is loaded, have the Chrome Inspector open. You will see some options load up under the "Remote Target" section. Select inspect from the first option. This will open up Chrome's developer window. From here, you can debug and see the console logs. When you make edits to your code, simply refresh this browser window and see your updates without having to restart the experience.

Default Shortcuts

  • Ctrl + x - Opens a modal that contains all of the console logs. This is very handy for troubleshooting a Live experience where the Chrome Inspector is not available.
  • Ctrl + a - This tool is experimental. It is a password protect Admin panel. Use this at your own risk. The default password is 492736.
  • Ctrl + c - Opens the communications indicators panel. The indicator icons show the on/off state for the existance of the avatar, internet, microphone state, avatar speaking.
  • Ctrl + r - Reloads the browser/UI.

Query Strings

Adding these query strings to the UI browser URL for their associating effects.

  • is_console=true - When the experience is running, the console logs are routed into the modal debugger feature. So when you are running the inspector the console log orgins are all going through Debugger.js. When using this query string it will stop routing the logs through the debugger. The result is that the console logs in the inspector/developer tools will show the source file from where the log was rendered.

Changing Project.json Files

We had a situation where we had multiple units where each unit had minor changes. To solve this we had separate project.json files then enabled the ability to run the experience using a chosen file.

This can be done in 2 ways:

  • In the index.html at the bottom of the page there is a <script> tag. Add config-json = "Project1.json". This will load Project1.json instead of project.json.
  • In the command-line parameters running PRSONAS_VIEWER.EXE add after that on the same line, json=Project1.json. This will load Project1.json instead of project.json.

Using The Event System

This framework uses a custom event system. All of the events are defined in window.tEvent.eventStr.

To trigger an event:
window.tEvent.fire("MY_EVENT_STRING", propertyToPass);

Listen to an event:
window.tEvent.addListener("MY_EVENT_STRING", function(evt, data){ });

Hammer

This framework uses HammerJS to manage UI interaction. This was for smoother handling on touch events. A common usage I recommend for button clicks is:

$("body").hammer().on("click", ".my-element", function(e){ });

This creates a delegate listener.

Caution

These are some things that can cause issues, so be aware of them.

  • In the project.json prsonas.timers.expiredEvent, make sure this event is different than the event used for the Walkaway state.

A Word About Your Content

The folder __static_ext is the framework. This content should not be altered. As the framework evolves, this will be the folder that would be replaced.

The folder content is where you want to do your development. The projectStyles.css, project.js, langCopy.json, and project.json are preloaded automatically. So they may be modified as you see fit.

project.json is unique in that this file is the configuration for your project and it is also an override to the framework configuration itself. project.json gets merged with and overrides the settings in __static_ext\dxp\json\core-config.json.

In content/js/project.js is a preloaded js class that you can use to create your own custom behaviors. It provides you with a simple set up and intialization to begin your customized experience.

Query String & Command Line Parameters

For convenience, there are a few command line parameters that can be used to change certain properties instead of needing to create multiple projects for minor changes. These parameters are set using a BAT file to load the PRSONAS_VIWER.EXE. Add a space, then add parameterID=value. NOTE: These parameters also work with query strings if you are developing ouside of the viewer for testing. Priority will be JSON configuration -> query string -> command line.

The following are framework only options that can be set:

  • json - This will change the JSON file that is loaded. By default, project.json is loaded.
  • interactiontimer - This is a number in milliseconds that is the time when an experience expires do to lack of interaction. To disable this timer set it to -1.

Intents And Using Generative AI (GenAI)

Intents, which are IDs associated with avatar animations are defined in the project.json file in the node sendKeyEvents using key/value pairs where the key is the event string that triggers the intent, and the value is the intent. For clarity, "sendKey" is also a term often interchangable with "intent".

GenAI applied to an intent is defined in the langCopy.json customIntents node. This uses key/value pairs where the key is the intent and the value is the dynamic text you want the avatar to speak. If the intent is not defined here, then when that intent is sent to the avatar, the avatar will simply try to speak first, the Knowledge Base answer response, If that does not exist then the avatar speaks whatever is preprogrammed/precanned.

There is no need to apply translations with genAI responses. When the avatar is set to a particular language, when the genAI text in English is sent to the avatar, the text will automatically be translated into the target language.

When working with sign languages, see Working With Multiple Languages.

Closed Captioning

The speech from the avatar is automatically received from the SDK when the avatar begins to speak. This copy is then displayed in the closed captioning element. This closed captioning copy can be changed if necessary in the langCopy.json ccCopy node that will overwrite the copy being displayed. You can also control how the copy scrolls when being displayed. Within the ccCopy there should be an additional layer that is the language ID per language being used. The syntax per closed caption entry would like like this:

"walkup": {
	"copy": "Welcome to the iHealth Assist Concierge. How may I assist you today?",
	"copyProxy": ""
}

In this example, the walkup closed caption would be replaced with this one.

copyProxy can be used for redundant copy for different intents. Below, is an example of the usage:

"walkup_1": {
	"copy": "Welcome to the iHealth Assist Concierge. How may I assist you today?",
	"copyProxy": ""
},
"walkup": {
	"copy": "",
	"copyProxy": "walkup_1"
}

This will intercept the closed captioning for walkup and use walkup_1 as the response. This will allow you to have multiple intents using the same copy if needed.

Knowledge Base

The Knowledge Base is used to interpret spoken word into something useful which returns an answer or a response. We use metadata to help communicate to the UX what to do with that response. If no metadata is returned then the UX will simply speak the answer returned. The intent used for the speech is the question replacing spaces with underscores and removing special characters.

There are a few metadata syntax that the framework handles:

action_ux

This metatag is used to identify UI actionable responses. If no action_ux is defined then it is assumed that there will be no UX impact other than a spoken response. Otherwise, The syntax is action_ux=contextID_actionId. Behaviors are defined in the project.json services.qaSpokenActions. There are 2 nodes that would represent the contextID that are defined, page and navigation. These will contain key/value pairs where the key is the actionId and the value is the event string that will be called.

  • page - This is used when the response is requesting to go to another page.
  • navigation - This is used for general navigation like "next", "back", or "home".

Example:

"qaSpokenActions": {
	"page": {
		"main": "EVENT_PAGE_MAIN_MENU"
	},

	"navigation": {
		"next": "EVENT_FOR_NEXT"
		"back": "EVENT_FOR_BACK"
		"home": "EVENT_FOR_HOME"
	}	
}

There are other actions that are defined internally that can be used.:

  • modal - Modal content is created through the compiled project HTML content. The name of the modal will be "m" + id.
  • closemodal - This will close any open modals. The id is not used.

action_speech

This metatag is used to identify spoken actionable responses. By default, genAI is used if this is not defined. If the response is action_speech=intent_intentID, the intentID would be the pregenerated intent you want the avatar to respond with.

Working With Multiple Languages

Changing A Language

The standard approach to changing the language is using the complex module for the language selection page which automates much of the process. This includes setting the language to the Avatar and the UI. When returning to the language selection page the language is reverted to the default language defined in the project.json.

You can manually set the language using the following code.

window.tEvent.fire(window.tEvent.eventStr.EVENT_APPLY_LANGUAGE, {
    language: "en-US"
});

UI Element Translation

There are few methods available to enable the translation of UI elements. All of which make use of element data attributes.

  • Image Swapping - When you are changing languages you may need to change the images. This is applied to <img>. Add the data attribute data-lang-img-id, the value will be this image name. Then you will place each copy of the image to its appropriate language folder in the content/images/. So for example content/images/_English or content/images/_Spanish. This will automatically swap the images when the language changes.
  • Manual Copy Translation - This method works without internet connection or connection to the avatar. You can change the language copy for UI elements manually by adding the data attribute data-lang-copy-id. The value is the ID that will be used to look up the language translation. You set those translations in the langCopy.json file under the languageCopy node. There are a number of manditory elements here that should be defined. These manditory elements are defined in the various predefined modules. Just add your own custom elements to the end of this object. The key will be the ID you defined in the data attribute data-lang-copy-id. Within the object you will set up key/value pairs where the key is the language, the value is the copy to be used for that language. Example:
"dxp_modal_qr_code": {
	"_English": "Please scan the QR code to learn more...",
	"_ASL": "Please scan the QR code to learn more...",
	"_French": "Veuillez scanner le code QR pour en savoir plus...",
	"_Spanish": "Escanee el código QR para obtener más información...",
	"_Icelandic": "Vinsamlegast skannaðu QR kóðann til að fá frekari upplýsingar..."
}
  • Automated Copy Translation - This method requires internet connect and for the avatar to be running; otherwise, the language will just use the default. To enable this feature requires an admin setting update. If this is set, then adding the data attribute data-auto-lang-source-id with the value of the language code that you are using the initially set the copy text. Then whatever is set in module's copy property will be used for translation. Behind the scenes, when the element is created, a data attribute data-copy is populated from the copy property and is then translated to the current active language and placed in the html element.

To force a translation on the fly for pages that have elements displayed after the initial page load, call...

window.oLang.applyCurrentLanguage();

Sign Language

Currently, we do not support genAI for sign language. A couple of features are there to support working with sign languge. When working in the Knowledge Base, genAI can be used by supplying an answer and can be initiated by supplying no action_speech metatag. For sign language you would want to add action_speechfallback: intentname. In the project.json under services.genaiFallbackToIntents is an array. This will include the language IDs that you want to use the action_speechfallback. The defaults are:

"genaiFallbackToIntents": [
	"_ASL",
	"_BSL"
]

Responsive Interfaces

Responsive interfaces is a challenge because the Viewer does not resize easily. There is a responsive overlay tool that can be used by including the query string or your batch file commandline param responsivetool=true. This allows you to dynamically resize the viewer for proper testing.

When using an interface that is responsive, remove all top, left, bottom, right params from your batch file commandline params.

You will also need to set the project.json avatar.isResponsive to true. What this does is there is a .avatar-placeholder element in the index.html. When isResponsive is true, when the viewer is resized, the avatar camera and position will adjust to fit within that box. The avatar will crop in those bounds so be aware of that. Use CSS to properly adjust the size/position of the .avatar-placeholder to get the responsive results.