My thoughts, experiments and ideas about Information Technology.

Vocabulary Notebook: From Idea to Reality in One Year

The 8th of September 2012 I started to work intensively in an idea I called Vocabulary Notebook. With that idea I was planning to solve a problem that myself and many other english students experience while learning vocabulary. It has already been a year since then and that idea is now a real platform used by thousands of students and teachers around the world. I must say that I’m quite proud of what we have achieved during this year, both as an entrepreneur and as an engineer. Although I don’t have much free time now, I have forced myself to write this post to summarize what has happened around Vocabulary Notebook during this year, from idea to reality. Since there are so many things that have happened, I’m afraid this post is going to be quite long, but I think that the anniversary deserves it.

Honestly, in this year I have worked a lot. A lot means an average of 80 hours a week… But this post is not about all the time I have worked and all the things I have learned. This post is about the first year of Vocabulary Notebook. It’s a summary of what have happened this year around our product, from the very beginning to the full platform we have today. I will probably publish another post soon in which I will talk about all I have learned during this year, doing some retrospectives about my decisions and describing all the tasks I have had to accomplish and all the roles I have had to play. But today I want to review what has happened to Vocabulary Notebook during this year in which it has been built.

The Idea

I had the idea of Vocabulary Notebook in my mind since 2011. We were living in Cambridge, UK and besides working remotely I was preparing the CAE Exam in an English School. Our teachers suggested us to use a paper notebook where we could write down all the words (phrasal verbs and idioms) that we were learning everyday; with a sample sentence and a definition. I had tried those notebooks before several times and they always failed for me. After a few days, the vocabulary was always messed. No alphabetical order, no topics order, difficult to edit, uncomfortable to carry… I tried it again, saving several pages for each letter this time, to have some alphabetical order. It failed again. They are uncomfortable. In this case the problem was that I had a lot of white pages in the middle; while editions were still messy and all topics were mixed… I asked my partners in the course and most of them had the same problems.

You could think that the problem is that Vocabulary Notebooks are not useful; that it’s better to do nothing. However, there are several studies [1] that prove the benefits of these notebooks in the Vocabulary Acquisition Process. Significant benefits. The base of the improvement is the personalization of the learning process. If you write your own definitions and sample sentences you tend to remember the word easily afterwards… You understand the word deeply and you won’t forget it easily. And it is true. If you read a definition from a dictionary, you will forget it quickly. If you create your own, first you have to understand the meaning and then you will generate your own definition. You have learned during the process. If you see the word again, you will remember instantly.

About UX in Modern Web Interfaces and Backbone.js: Choropleth Maps

I would like to start this post anouncing that this month I am leaving Arte Consultores Tecnológicos, after one year and seven months working there as an Analyst/Programmer (Front-end Engineer). I have made this decision because right now I need to devote myself exclusively to Langproving, which is in need of my full atention at this critical moment in which our MPV is finished and we have to market it to validate our business model.

In this period of time working at Arte Consultores, I have improved significantly as an engineer. I have learnt a lot. Fortunately, I have had the great pleasure to work closely with my workmate and friend Axel Hernández, who I admire for his ability to constantly learn new technologies and for being always aware of all the new trends in software development. I’m proud to say that I have learned a lot from him. I would like to think — actually, I hope — that he has also learned something from me too while we have been working together.

One of the most valuable things I learned working at Arte was related to writing modern and usable web interfaces, not only with CSS3 animations or HTML5 new attributes and tags; but also with Javascript. In the last few years Javascript has grown up and become one of the keys of the evolution of the web. Many Javascript frameworks have emerged, ranging from those regarding DOM manipulation (jQuery) to others built to reproduce the model-view-controller (MVC) pattern in the client side, such as Backbone.js. For more than a year I have been using Backbone to create Single-page applications that provide a quick response to user interaction, without the need to re-render the full page from the server as we were used to; thus improving sifnificantly the user experience (UX).

Choropleth Maps

In the following example we have a choropleth map from Stat4you in which the unemployment rate in the different regions of Spain is shown. The darker the colour, the higher the unemployment is. I have picked this dataset randomly, but it is worth mentioning that it shows a sad reality of Spain’s economic crisis.

We can see different elements in the map with which several actions can be performed. Firstly, we have the map itself, which happens to be a SVG vectorial image. Secondly, we have a zoom bar that has two different buttons (+ and -) and also has a draggable element that allows us to decrease or increase the zoom — as you may notice, it must be somehow synchronized with the + and - buttons —. Aditionally, we can use the mouse wheel to zoom in and out (as we can do in Google Maps). We can also click on a particular region of the map to zoom in and center the display on it; and we can click in the button above the zoom seciton to zoom out, going back to the original zoom and position of the map. We have also a leyend section, which indicates the ranges covered by each colour in the map. Finally, we have the ranges bar, which has a draggable element in the middle that can be used to increase or decrease the number of ranges to be used. It is also possible to click on the bar itself to increase or decrease the number of ranges. Clicking in the right side increases by one the number or ranges and clicking in the left side decreases the number of ranges in the same amount (the draggable selector will be moved automatically).

As it can be seen, there are many elements that interact with each other. Since this is a Backbone application, everything runs on the client side. The server only loads the initial data, sends it to the client browser and then it is no needed any more. Only if the data to be displayed changes, an AJAX call is made to the server to bring new data, but not to reload the page. Anyway, this AJAX calls are beyond the scope of this post. We will focus on the interaction among the described controls of the map, which are mapped to diferent views in our MVC pattern implementation, all of them linked by a model.

In this new image we have just moved the ranges selector two positions to the left. This has caused automatically two changes in other elements, besides the changes in the ranges selector itself. On the one hand, the leyend has changed. The ranges have been re-calculated and now they are just three, instead of the initial five. Moreover, the map has been redrawed and some regions have changed their colour, since now the ranges are different and the unemployment rates of the regions are grouped diferently.

In this case, we have clicked in the autonomous city of Ceuta. When clicking in a region, as well as when we use the mouse wheel, the zoom changes. We have zoomed in to the best level of zoom in order to see the selected region properly (this is done automatically when clicking in a particular region). Moreover, the map is now centered in the selected region. As we can see, besides the changes in the map, the zoom controls have also changed. The selector is now stuck to the top, stating that the maximum level of zoom is being used.

As you can see, there are several actions that affect different elements of the maps. All of them are being performed in the client side, in the user’s browser, thanks to Javascript and, in this case, thanks to the use of Backbone MVC Framework too. To display each element we have used different Handlebars templates. For the special case of the SVG maps, it is worth mentioning that they are being displayed by using another Javascript library called D3js, which provides poweful visualization components and tools for manipulating documents based on data.

To clearify how an MVC pattern can be implemented for this sample case by using Backbone framework, the following diagram has been made.

In the diagram we have represented the User Actions, the Map Model, the different Views for each section and a Map class that could be integrated in the MapContainerView if desired. Aditionally, the HTML / SVG code to be displayed in each case is stored in Handlebars templates and it is loaded in the “render” methods of its corresponding view. During execution, every action of the user triggers an event that is captured by the Map Model. Then, the model preforms different actions depending on the behavior of the user, and once it finishes, changes in some of its attributes are made. When the model is changed, new events are fired. Each view is listening to the events triggered by the model that are associated with it, thus being able to re-render itself if needed. Moreover, in case some mayor changes that require all elements to re-render are made, the render method of the MapContainerView will be called, which will execute each element’s render in turn.

In summary, as I mentioned before, Javascrip MVC frameworks are quite powerful and lead to faster websites in which no communication with the server is needed in order to perform changes related to user actions, unless those actions require new data to be received. Definitely, the development of web interfaces combining the power of Javascript (jQuery, Backbone…) with the new animations and attributes of CSS3 and HTML5 provide us the great technical tools that have led to a new revolution of web interfaces. Moreover, if we combine these powerful tools with the implementation of design patterns such as Responsive Web Design, Graceful Degradation and Progressive Enhancement while following a User-centered design, we can achieve an amazing user experience and a great satisfaction from our customers.

Improving the UI to Achieve a Better UX: My Experience in Stat4you

I had this article started but uncompleted since September 2012, and as a New Year’s Resolution for 2013 I decided to finish it.

Back then, I had been working for some months as a Front-end engineer in Stat4you. In the early summer we were in a hurry to finish the alpha version of the platform, so we released it without paying too much atention to the details. A feature that was included in this release were the filters for the charts, by means of which users were able to change the positions of the dimensions and also to enable and disable the categories of those dimensions, thus changing the chart being displayed.

The images below are screenshots of the filters initially released in Stat4you:

The image above shows the filters available for the tables in the initial alpha version of the platform. The blue elements represent the containers of the dimensions which, in this case, have been also placed in a particular way to help the user understand the position in which every dimension will appear. The white elements represent the dimensions whilst the orange ones represent the categories of each dimension.

In this image, the categories of one of the dimensions have been unfolded, this way the user can see all the categories available for a particular dimension, not only the previewed ones that can be seen without clicking in the right arrow. Also, it can be seen that the “Periodos” dimension has been moved to fixed (by doing drag and drop). This means that this dimension has been set to a specific category, which by default was the first one.

Finally, there were also filters for the charts. Opposed to the table, in this case all the dimensions were displayed horizontally. The white text over the blue background indicated the position in which the dimension was going to appear if placed there (in the x-axis, in the columns…).

As you could guess, this filters were kind of messy for the users. It was a quick design created in a hurry by us (developers) to launch the initial version of the product; so we made the mistake of not taking into account the user experience as much as we should have, although it was a MVP and was going to be improved in the near future.

The days following the launch of the alpha version of Stat4you we had some feedback from visitors, and a portion of the negative feedback was regarding the filters, which appeared to be (as you can guess) confusing and difficult to use, causing a poor user experience. Hence, we decided to improve the user interface to achieve a better acceptance.

To improve the filters we run some usability tests. In my case, I asked for the favor to some friends and family and started the experiment. In this experiment I asked them to access to some datasets within the platform and, once the dataset was accessed, I encouraged them to change the displayed chart or table to achieve a particular visualization (fixing dimensions, disabling some categories of other dimensions and changing their positions). As other users had reported before, the results were poor. It was difficult for them to achieve the changes I requested, but I got tons of information about the improvements needed: such as a search box for the categories, a better indication of the positions in which dimensions could be placed or a visible indication of which was the selected category in the fixed dimensions.

Once these improvements were identified all we had to do was to develop them,knowing that those functionalities were the real needs of the users.

The result of these changes can be seen in the following images:

In this first image, the main changes were the labels indicating the positions in which dimensions could be placed and the width of the fixed position box, that now occupied the 100% of the available width.

In this case, some changes can be seen: For the fixed dimensions now the selected category is visible at every moment, preventing the users from unfolding the dimension to know which one the selected category is. In addition, a search box has been introduced to search within the categories of a dimension, making it much easier to find a particular category. Moreover, the ‘x’ that was placed at the top right position of the unfolded box was removed and replaced by an ‘accept’ button at the bottom right corner. In our experience, this change that forced users to accept the changes in order to close the unfolded categories, helped them to realize that the changes made are always going to be persisted. Before this change was made, some users had the sensation that when the ‘x’ was clicked, it was going to undo the changes.

For the charts, the main change were the labels and shapes of the position boxes. The new interface seems to make the available options of edition easier to understand for users. Besides, it mantains greater uniformity with the tables filters.

After these changes, we obtained positive feedback from users, and in the usability tests conducted afterwards, the user experience improved significantly. Firstly we had developed a UI without testing it with real users and when we released the product, the user experience with the filters was poor; we had to conduct some tests once it was in production, detect the problems users were having and then rewrite the interface and re-launch the product with the improved filters.

In conclusion, what I learned back then was something that I already knew, but sometimes with the rush and the misunderstandings of what a MVP actually is, we tend to forget: When designing user interfaces it is really important to prototype first, do some usability testings with the prototype and improve user interfaces by means of users feedback. Then, once these first steps are finished, it is time to implement the tested solution. This way, it will take more time to launch, but it ensures us to achieve a good user experience since the launch day, avoiding the extra time and cost of rebuilding the feature afterwards. It could seem faster to design and implement a simple UI without usability testing and user feedback, but it will have the drawback of causing a worse user experience and, in the long run, it will probably cost more (time and money) in order to rebuild the UI to achieve a better UX.

The Front-end Engineer Role

The Front-end engineer is a relatively recent role in software development, but its importance is growing everyday as the importance of Javascript does. I’ve been working as a front-end engineer in the Stat4you project since January and in this article I will try to describe this role, emphasize its importance and highlight the differences that the introduction of Javascript has brought to the development of the front-end of web applications.

I would like to introduce this article with a quote from the human-computer interaction expert Jeff Raskin that summarizes quite clearly the importance of software interfaces from the perspective of users. It reads: “As far as the customer is concerned, the interface is the product”.

Although the quote above seems quite obvious, often developers and managers tend to forget or underestimate the importance of user interfaces, blinded by the complexity of the back-ends of the applications. This blindness has led many projects to failure, but fortunately this is starting to change.

In order to continue highlighting the importance of user interfaces, there is another quote from Linus Torvalds on this topic which I also fully agree with: “In many cases, the user interface to a program is the most important part for a commercial company: whether the programs works correctly or not seems to be secondary.”.

Quite often, traditional developers teams give priority to the development of new and complex features (which mostly will never be used by the majority of end-users) instead of spending the time in the creation a friendly and easy to use interface. This frequently results in a hard-to-use application with a lot of functionalities that will never be used due to the poor user experience caused by its unfriendly interface. Thereby, many projects meet with failure due to usability issues rather than lack of functionalities, as Linus Torvalds said.

But… What is exactly the interface? And, who develops it? For many years there hasn’t been a figure in the development process dedicated exclusively to interfaces. Usually there was only the general concept of developer or programmer, who sometimes coded the business logic or the services, while in other occasions had to control user events or choose the color of the buttons and the way the information had to be displayed.

It is true that it has been years since there is also a role related somehow with interfaces. It is called the designer, but usually this role is related to someone with an artistic background, most of the time it is someone who works fine creating images with Photoshop, but with little or none technical knowledge about HTML, CSS and Javascript. This kind of professional cannot be considered a Front end engineer. It is just a graphic designer.

Working on this article I found an interesting categorization done by Rob Larsen in which he describes three profiles of front-end engineers: “the creative” (who would be probably a designer with HTML and CSS skills), “the coder” (who would be a javascript programmer, also strong at CSS and HTML but probably weak at design) and there a last one, named “the core” (this would be somewhere in the middle of the other two).

The profiles defined above are rather wide, but I find them quite accurate. In my case, due to my programmer background and these last months focused on javascript I fit probably better into “the coder” although I see something of me in “the core” profile. Probably I was in this profile years ago, before studying at university, when I was interested in the web in general and I started learning both design and programming by myself. Then, at university I started to learn more about programming and I became more “coder”.

Reading another post of Rob Larsen in which he describes the different front-end roles, I found his description of a “Front end developer”. He defines it as someone who is “responsible for HTML, CSS and JavaScript templates and prototypes at the beginning of development and remains heavily involved with the integration and QA phases of a site build. This person is solely focused on the presentation layer. While one might occasionally help out with tasks on either the design side or the server side, 95% of the work done in this role is in the narrow band defined above.”. He also says that this role is for a “core” or a “coder” who is happy to focus in javascript as his main programming language. I think that in the project I’m working on right now I fit in this description of front-end engineer fairly well.

About the old concept of designer, Rob Larsen says: “The general trend of the front end engineering spectrum is expanding away from this type of role” mainly due to the importance of javascript.

In the image above, we can see the distribution of tasks between front-end and back-end in a web application. In the case of the front-end I have separated it in two columns: on the right side we have the traditional web application (where the presentation code is in the server) and in the left side we have the modern concept (with javascript controllers in the client side).

Nowadays with the growing importance of Javascript and the single-page applications, the controllers of the web applications are being moved to the client using javascript MVC frameworks such as Backbone.js, with the goal of providing a more fluid user experience akin to a desktop application. In our case, we have thinned the server side significantly with this approach. The server side provides a REST API from which the client consumes the services. The client sends AJAX requests to the API and the server sends JSON responses with the information required.

Thus, in our case the information processing and the built of the web interface are done in the client side by the front-end engineer, using Javascript for the behavior and HTML and CSS for the markup and styles. For us this has been possible and maintainable thanks to the help of many frameworks, libraries and tools such as Backbonejs, Underscorejs, jQuery, D3js, Highcharts, Jasmine, Handlebars, Less and the Google Chrome Inspector for debugging. Surely, I would write about some of them in more detail in future.

So, in conclusion, front-end engineering is not designing, even though the design is a part of the front-end of an application. A front-end engineer must be someone focused on the presentation layer, mainly responsible for HTML, CSS and JavaScript templates and prototypes even though might occasionally help out with tasks on either the design side or the server side. With the growing importance of Javascript and the help of frameworks and libraries the controllers are being moved to the client side and being written in Javascript, leading the front-end engineers to focus on this programming language.

Giving Octopress a Shot for This New Blog

I’ve been trying to set up a new Blog for a while, but to use a nice excuse, I haven’t had enough time to do it. On Friday I was talking with my workmate Axel about Octopress and it seemed interesting to me. I have just finished the installation and deployment of it using GitHub and it has been rather easy.

While I was reading about Octopress, I found this post where the author writes about the importance of static rendering for webpages and blogs at present. Nowadays the most popular CMS systems store post and pages in a database and then they build the pages from the database on every request. Although it is also true that there are some cache plug-ins to improve this behavior, he states that this traditional implementation is inefficient. This idea reminded me that two years ago, when I built my own CMS for personal websites Sigecu CMS, I had this idea in mind, so what I did was to generate a static HTML page every time the user makes a change in the website, avoiding this problem. I would try to write some posts in the future about Sigecu.