Project Report

A few years ago, I purchased my first Raspberry Pi, a tiny computer that can be easily programmed and configured in a variety of ways.  My passion for creative implementations of technology was immediately ignited, and I quickly convinced some friends to buy them as well.  Before long, we were all trading stories of creative and helpful ways to use them in our daily lives.  One particularly exciting use we found on the internet was a “magic mirror”, a device which looks like a normal mirror but which hides an LCD screen behind the glass.  These mirrors use small computers inside the frame to control what information is shown on the screen – a perfect use for a Raspberry Pi.  I started researching how to build one myself, and in doing so realized how useful smart mirrors like these would be to most people.  Unfortunately, though, building and configuring a magic mirror is an involved process which can be too daunting or time-consuming for most people – with programming the display correctly a particularly complicated step.  A commercially-available smart mirror would help solve this problem, but it too would need to include simple, user-friendly customization software in order to succeed.  After I recognized the emerging need for a software solution to this problem, I decided to make it the topic for my thesis work.

More and more modern appliances and devices are being designed with integrated “smart” capabilities, but smart mirrors like the magic mirror have yet to penetrate the market.  One problem that limits a potential smart mirror product is the lack of software for designing and configuring a smart mirror’s layout.  With my thesis, I wanted to see if a designer interface prototype could be created that would aid users in creating and customizing a mirror layout.  Furthermore, would an interface capable of managing the necessary complexities of all possible layouts be simple enough for casual users?  And if a smart mirror product were to be developed for the consumer market, could the prototype serve as a possible basis for a consumer-facing interface?

I chose this problem for a number of reasons.  First, I was personally interested in the problem as a result of my experiences with Raspberry Pi projects.  Second, I recognized an information problem that could be solved with interface design.  Third, while there seemed to be a clear need for software that made it easy to customize a smart mirror layout, I was unable to find any existing tools for the task.

Whereas other methods of creating or customizing a smart mirror’s layout rely on a user either coding the layout completely themselves or manually editing configuration files, I decided to pursue a simple visual interface solution.  My prototype would include the following elements:

1. A number of “modules”, each displaying different information, which could be selected by the user and arranged in different positions in a grid-based layout. Examples of modules include a clock and the local weather.
2. A saving function allowing a user to save specific layout designs and load them at a later time.
3. Module-specific options, allowing a user to configure the modules to specific settings.
4. Color and font selections, allowing a user to further personalize the visual appearance of the mirror layout.

During the early stages of developing my thesis proposal, I was unable to find any examples of tools for designing smart mirror interfaces.  While my research did uncover a few examples of magic mirrors built by hobbyists using raspberry pi computers, the software controlling them had all been manually coded.  As a result, I formulated a plan to build my prototype as a website, using publically available APIs to power the different modules.  My thesis proposal reflected this plan.

However, early in development a man named Michael Teeuw, a magic mirror creator I had become aware of during my research, published a new platform for magic mirror interfaces called MagicMirror².  This software can be easily installed and, like my planned prototype, uses modules to display different types of information.  The problem with the MagicMirror² software, unfortunately, is that customizing the layout requires manually editing a configuration file.  This means that some amount of coding knowledge is still required to create a magic mirror layout.  In addition, the number of specific settings and modules means that errors are possible even for an experienced programmer editing this file.  Finally, while the software includes well-written readme files and user instructions, creating a custom interface still requires first becoming familiar with these materials.

After installing and using the software myself, I decided to modify my prototype to make use of the MagicMirror² platform.  Rather than building the prototype as a website, I formulated a new plan to develop it using the Electron framework, creating a prototype that would allow a user to quickly and easily create a magic mirror layout that would then be formatted and saved to a MagicMirror² configuration file.

I decided to build a magic mirror to house serve as a physical display of the prototype in action.  This required first obtaining an LCD monitor from which I could remove the screen, a sheet of semi-reflective plastic to serve as the mirror’s surface, a Raspberry Pi computer, and materials to construct a frame.  I created a workspace in my basement and began the process of sawing, drilling, sanding, and painting the wooden frame.  I mounted the reflective plastic and LCD screen inside, then the Raspberry Pi.  After attaching the HDMI and power cables, I affixed a back panel to the frame.

Because I would be building the Magic Mirror Designer prototype in Electron, which I had never used before, I educated myself on the platform during the process of building the magic mirror described above. I found that Electron was simple to learn, and by following a few online guides I was able to begin creating apps in Electron quickly.  In addition, the decision to develop with Electron was helpful for two reasons:

1. Electron is powered by traditional web technologies, meaning it was still possible to create the prototype using Javascript, HTML, and CSS, as was my original plan.
2. Electron applications are cross-platform and are able to be packaged as desktop apps, meaning more potential uses for the prototype could be explored.

After considering how best to implement the planned elements of the prototype using Electron, I decided upon the following development framework:

1. The program opens windows for configuring settings or loading saves.
2. The program saves inputted settings as files which can be read by the MagicMirror²
3. The program “loads” saved configuration files by moving them into the location read by the MagicMirror²
4. The program launches the MagicMirror² software on command.

To achieve this, I duplicated the core MagicMirror² software and modified the electron.js file, which is run when the software is launched, to open the Magic Mirror Designer window.  The following code shows the function called by the program when it is first loaded:

function startDesigner(){
  //Create the info window and hide it from view.
  infoWindow = new BrowserWindow({width: 500, height: 500, frame: false,
  maximizable: false, parent: designerWindow})
  infoWindow.loadURL('file://' + __dirname + '/../moduleInfo.html');
  infoWindow.hide();
  // Create the designer window.
  designerWindow = new BrowserWindow({width: 1000, height: 750, fullscreen:
  false, maximizable: false, frame: false});
  // and load the index.html of the app.
  designerWindow.loadURL('file://' + __dirname + '../../designerIndex.html');
  // Set responders for window events.
  designerWindow.on("closed", function() {
  designerWindow = null;
  });
};

From there, I created HTML, CSS, and Javascript files, as if I were building a webpage. I built dropdown menus for the available modules and positions, with the module-specific options coded to be written contextually using Javascript as each module is selected. To launch the magic mirror, the program first saves all added modules and settings to config.js and custom.css files, which are read by the MagicMirror2 software as it is launched.
In order to implement saving and loading of mirror layouts, created folders for saved Javascript and CSS configuration files. When a layout is saved, the user inputs a save name which is used for the saved files. When loading, the program reads the contents of the save folders to display all saved designs. When the user selects one, the Javascript and CSS file with that name are copied to the config.js and custom.css files read by MagicMirror2. The code below shows how saved layouts are loaded into the saved designs table:

//read contents of saved configurations directory
function readSavedConfigs(){
	fs.readdir('./resources/app/config/savedConfigs', (err, savedConfigs) => {
		drawSavedConfigsTable(savedConfigs);
	});
};
//draw table of saved configurations
function drawSavedConfigsTable(savedConfigs){
	for (var i=0; i<savedConfigs.length; i++){
		var t = savedConfigs[i].replace(/\..+$/, '');
		var tableRow = configsTable.insertRow(i);
		var tableData = tableRow.insertCell(0);
		tableData.setAttribute("onclick", "savedConfigSelect(this)");
		tableData.innerHTML = t;
	};
};

For the visual design of the prototype, I created a set of wireframes to serve as design templates.  As I developed the prototype, I followed these wireframes to create the basic structure of the program windows.  In order to create the wireframes, I thought about how a potential user would approach creating a custom mirror layout, what actions he or she would take, and in what order.  My typical user actions would look like this:

For a user creating a new design, the actions necessary are shown in blue.  Steps 1-4 can be repeated for as many modules are desired.  For a user wishing to load a previously-saved design, the necessary actions are shown in red.  These users should be able to bypass the other steps entirely.

Following this plan, my design for the prototype splits the main window into sections.  At the top of the window, a toolbar contains the program name and a quit button, as well as a button for opening saved designs.  By keeping this button on the main toolbar, it is easily accessed and ever-present.  This button opens a window displaying all saved designs, allowing a user to complete the red actions and launch the magic mirror with a loaded design.

For users creating new designs, the left side of the main window is the configuration area.  This section contains all the necessary elements for the blue actions. The right side of the main window is the preview area, which shows what modules have currently been added to the design, and in what positions.  As each new module is added to the design, it appears here.  This area also includes a reset button, which removes all added modules from the design.  At the bottom right of the main window, the launch area contains the main launch button for starting MagicMirror² with the completed design.  Finally, this section also holds a save button, allowing a user to save a design under an inputted name.

I began my first round of user testing after all functional elements had been added to the prototype, but before the final visual design had been implemented.  This was to allow feedback from user testing to inform the final look of the program.  I read to each of the three participants a short introductory text, which served as basic information on magic mirrors and the function of the Magic Mirror Designer.  The participants were then asked to complete a series of ten tasks using the prototype.  The first three tasks asked the users for verbal responses to questions about the prototype, while the following seven tasks were timed actions for the users to complete.

Following this round of user testing, I analyzed the responses and determined the following:

1. Users were able to navigate the program, but sometimes had difficulty finding specific buttons or elements when needed.
2. When first launching the program, it appeared mostly empty to users.
3. Users sometimes paused on tasks to customize their design, outside of required tasks.

To improve on the first point, I shaded the module configuration and preview areas.  This was done to further segregate the different sections and help a user to quickly focus on a relevant area in order to find a specific element.  This also had the effect of making the program look more visually balanced when first launching it, thereby addressing the second issue above.  I made no changes to the program in relation to the third point, as it seemed it was a sign of a positive response from users.

After I completed the final design, as informed by my user testing, I asked the same participants from the first round of testing back for a second round.  This round was more informal, as I did not ask the users to complete any specific tasks.  Instead, I had them fill out a System Usability Scale (SUS) survey.  This survey was obtained from www.usability.gov, and asks users to respond to a series of ten statements with a number from one to five, representing whether or not they agree with each statement.   The scale evaluates a system’s general usability and outputs a score from 0-100 for each participant’s response.  According to the www.usability.gov information, any score above 68 is considered above average.  While I only tested three participants, each score was well above—with an average score of 87.5, indicating that the prototype was in a completed state.

During development of Magic Mirror Designer, some aspects of the planned design had to be curtailed or modified:

1. The planned website implementation of the prototype was not pursued.

This was necessary after it was decided to use Electron and incorporate MagicMirror² into the prototype.  There were a number of advantages to doing this, and the benefits of the change outweighed the cost.  Primarily, the switch meant a much more robust layout format, with greater positional complexity and the ability to dynamically stack multiple modules in each position.

2. The original preview pane, featuring a visual representation of the final design, was changed to a numbered list of modules and positions.

This was the main cost of the decision to use Electron.  The original website design would have called for a much more simple, grid-based layout than what is possible with MagicMirror².   This means that the design could have been represented visually relatively easily by also creating a grid layout in the preview pane.  However, MagicMirror²’s layout has more possibilities, and allows for stacking modules in each position.  A visual preview pane was attempted for this layout, but the complexity of accounting for all possible design combinations created too high of a resource cost, and a simpler preview design was chosen.  To retain the ability to preview the layout visually, a preview button was added which launched the mirror software in windowed mode.

There were also other elements which exceeded expectations or were expanded from the initial plan:

1. The allowed modules are more complete and customizable than originally possible.

Because my original plan required me to create each of the modules individually, my idea was for each of them to have only very basic functionality.   Using the MagicMirror² platform not only meant that coding the modules was no longer necessary, but also provided complete modules which were only slightly modified to fit the prototype’s needs.

2. The system of saving custom layout designs was developed much easier than expected.

One of the biggest challenges known at the time of my proposal was the difficulty in implementing a system of saving a user’s selection for later loading.   The simplest solution was to save the information in the browser’s cache, but this would have meant the saves were easily lost.  Saving the information to a server would have been more effective, but something I inexperienced in implementing.  By using Electron to develop the prototype instead, I was able to easily access the system’s file structure and save the information locally.  Ultimately, I believe this provided for a much more user-friendly saving system.

In any future development of Magic Mirror Designer, I would be interested in pursuing two major avenues. First, I would like to create fully packaged versions of the program for several platforms, in addition to the unpackaged version available now. It is difficult to package an application in Electron for a platform on which it was not developed. Because I developed the prototype on a Windows machine, I have only been able to successfully package it for the Windows platform. By packaging them on other platforms, I hope to get more working packaged versions.

Secondly, I would also like pursue incorporating support for additional modules into the prototype. One advantage of the MagicMirror2 platform is the support for community-created modules. Some of these modules, for example a live video module, would be interesting to support with the Magic Mirror Designer program.

I believe the development of the Magic Mirror Designer has been successful.  The participants in my user testing all said they had positive experiences with the software, and showed a clear interest in experimenting with creating different layouts.  The amount in interest shown by the participants also indicates that there is a real interest for magic mirrors, especially if the software for customizing them can be simple and user-friendly.

My hope is that this prototype can make it easier for anyone wanting a magic mirror of his or her own to install and customize the necessary software.  Furthermore, the prototype could serve to inform the design of a future smart mirror consumer product’s software platform.  I am happy with the final version of the prototype, and would like to thank my advisor, Jay Taylor-Laird, for his guidance and support, as well as all my user testing participants.