Skip to content

Posts from the ‘Programming’ Category

21
Aug

Load Many Images in a Single Request (Optimization)

Some pages on GoToQuiz.com generate many requests to retrieve many thumbnail-sized images. Each image file is only 1 – 5 KB in size. I wanted to reduce the penalty associated with making numerous requests, so I came up with a way to combine these into a single request.

I needed a solution to retrieve an arbitrary number of images, which may vary with each pageview. I considered generating an image sprite on the fly, but this approach came with several problems:

  • Constantly invoking image functions on the server, adding more overhead
  • Image display would require using background-image and CSS manipulation
  • Image display would also depend on meta data about each image, to correctly set the necessary CSS values
  • Images may be different heights and widths, further complicating sprite generation

So instead, I went with returning the image data base64-encoded in a JSON format. It works like this. The HTML will have elements each holding the URI of the image as a data element rather than in <img src=”…”>.

<div class="gImg imgSwapHiPri" data-imgpath="/upic/2017/21/Mn8rrw.T1.jpg"></div>

In this case, it’s a certain kind of thumbnail styled with .gImg to have a specific height, width, and generic default background. The .imgSwapHiPri class is used to tag all elements to have their images swapped in.

And here’s the jquery plugin I wrote, to be called on pageload:

(function($, dataAttr) {

    $.lazyImage = function($swaps, alt) {
        $swaps = $($swaps);

        var thumbs = $swaps.map(function() { 
                return $(this).data(dataAttr);
             }).get();

        if (thumbs.length !== 0) {
            $.get('/img/image-set.json', {"paths[]": thumbs}, function(json) {
                $swaps.each(function(i, el) {
                    if (el.tagName === 'IMG')
                        el.src = json.base64[i];
                    else
                        el.innerHTML = '<img src="'+ json.base64[i] +'" alt="'+ (el.getAttribute('data-alt') || alt || '') +'">';
                });
            });
        }
    };
        
})(jQuery, 'imgpath');

Very simple. Invoked with $.lazyImage(‘.imgSwapHiPri’);. You can pass an optional alt attribute value, or optionally set a data-alt attribute per element. This script checks if the element to be swapped is an IMG tag; if so, set it’s src attribute. If not, set its innerHTML to an IMG tag. Either way, the image data is set to be the base64-encoded image returned by /img/image-set.json.

Let’s take a look at what’s happening in /img/image-set.json’s PHP code. I’ve got some framework boilerplate in the actual script, but the main bit is this:

const WEB_ROOT = '/path/to/public_html/';
$paths = $_POST['paths'];
$b64 = array();

//fill an array with the base64-encoded images        
foreach ($paths AS $path)
    $b64[] = base64EncodeImg($path);

header('Content-type: application/json');

echo '{"base64":';
echo json_encode($b64);
echo '}';

//function to convert image path to base64 data
function base64EncodeImg($path) {
    //first ensure the image path contains only legal characters
    if (preg_match('@^([-_/a-z0-9]+\.)+(png|jpg|gif|svg)$@i', $path)) {
        //absolute path to file
        $path =  WEB_ROOT . (strpos($path, '/') === 0 ? substr($path, 1) : $path);

        if (file_exists($path)) {
            $type = pathinfo($path, PATHINFO_EXTENSION);
            $data = file_get_contents($path);
            return 'data:image/' . strtolower($type) . ';base64,' . base64_encode($data);
        }
        else 
            return '';
    }
    else
        return '';
}

The data generated by the code above is a simple JSON file containing an array of base64-encoded images. Critically, it is generated in the same order as the elements in the HTML. This allows the jquery plugin to easily match base64 data with the appropriate element just by iterating over them.

And that’s really all there is to it. When the pageload event fires, the jquery plugin is called and makes the request for, say, a dozen images. The image data is returned in one JSON response–typically under 30KB in my case with JSON gzipping turned on (mod_deflate)–and it is fast.

BONUS:

If you have alternate high resolution versions of images for high DPI screens, this modified version of the above jquery plugin has your back.

(function($, dataAttr) {
 
    $.lazyImage = function($swaps, alt) {
        $swaps = $($swaps);

        var hdpi = window.devicePixelRatio || 1 > 1.3,
            thumbs = $swaps.map(function() { 
                var $this = $(this);
                return hdpi ? $this.data(dataAttr + 'H') || $this.data(dataAttr) : $this.data(dataAttr);
             }).get();

        if (thumbs.length !== 0) {
            $.get('/img/image-set.json', {"paths[]": thumbs}, function(json) {
                $swaps.each(function(i, el) {
                    if (el.tagName === 'IMG')
                        el.src = json.base64[i];
                    else
                        el.innerHTML = '<img src="'+ json.base64[i] +'" alt="'+ (el.getAttribute('data-alt') || alt || '') +'">';
                });
            });
        }
    };
        
})(jQuery, 'imgpath');

In this case, the plugin checks if the screen is high DPI and checks each element to see if there is a data attribute called “data-imgpath-h”. If so, it uses this instead of the value of “data-imgpath”. The value of data-imgpath-h is the path to the high resolution version of the image.

22
Jan

A Super-Lightweight jQuery Toast Plugin

Looking for a lightweight toast jQuery plugin?  Here’s a super simple one I wrote for this site, called gtqToast. It minifies to just 520 bytes.

TOAST PLUGIN DEMO

(function($) {
    'use strict';

    var $toastList = $('<ul id="gtqToast"></ul>').appendTo('body');

    //register the plugin
    $.fn.gtqToast = function(options) {

        var settings = $.extend({},
                                {fade:      500,
                                linger:    4000},
                                options),
            $toast = $('<li style="position:absolute;visibility:hidden;"></li>')
                            .appendTo($toastList),
            $container = $('<div></div>')
                            .html(this)
                            .appendTo($toast),
            height = $toast.height();

        $container.hide();
        $toast.css({display: 'none',
                    position: 'static',
                    visibility: 'visible',
                    height: height})
            .slideDown(160, function() {
                $container.fadeIn(settings.fade)
                    .delay(settings.linger)
                    .fadeOut(settings.fade, function() { $toast.remove(); });
            });
    };

    $.gtqToast = function(text, options) {
        $('<span></span>').text(text).gtqToast(options);
    };

})(jQuery);

It needs this bit of CSS as well (note that I assume you use a CSS reset, otherwise you’ll want to add properties like list-style: none):

#gtqToast { position: fixed;
            bottom: 10%;
            z-index: 102;
            text-align: center;
            width: 100%; }
#gtqToast div { display: inline-block;
                background: rgba(0,0,0,0.75);
                color: #fff;
                font-size: 15px;
                font-weight: bold;
                padding: 14px 20px;
                border-radius: 20px;
                box-shadow: 3px 3px 9px 1px rgba(0,0,0,0.3);
                margin-top: 8px; }

What the plugin does: inserts an empty ul (unordered list) tag into the DOM with a fixed bottom position 10% up from the bottom of the viewport. This list will hold the toasts. You can trigger a toast in two ways.

// #1, call on a jQuery object
$('<span>This is my toast</span>').gtqToast();

// #2 call with text
$.gtqToast('Another toast, yay!');

These calls will add the toast to our list and fade them in. If you look at the plugin code, I actually add a div inside the li to contain the toast message. Plus you’ll notice some weird CSS changing. I did this to account for multiple toasts becoming visible at the same time. What will happen is, existing toasts will slide upward when a new toast is added, then the new toast will fade in.

There are only two settings you can change on this basic plugin: fade is the fade in/out speed, and linger is the length of time the toast will stay on the screen. The defaults should be fine for most cases.

Feel free to use this if it fits your needs!

13
Dec

New Project Launched: Spectacular.Gift

It began like this: Amazon.com’s vast product catalog contains many clever and unique items, the sort that you may not know you wanted until you’ve heard of it. Alternately, these items might make an ideal gift when shopping for the person “who already has everything”. So I figured it would be a neat idea to curate a collection of these items and build a gift recommendation site around them. Doing so would allow me to explore some new server-side technologies and help keep my skills fresh.

Technologies used:

  • Ubuntu VPS from Linode
  • Apache2 HTTP server
  • Apache Tomcat 7
  • Apache ActiveMQ
  • Spring 3.1 with Spring MVC
  • PostgreSQL 9.3
  • HTML5 + CSS3
  • jQuery, flot, TinyMCE
  • Java libraries such as Joda Time, Jsoup, Jackson, Logback, and Commons DBCP
  • Amazon Product Advertising API
  • Reddit REST API

To get some data populated in the database as a starting point, I set up a scheduled task to pull data from several Reddit forums where Amazon links are shared.  Reddit conveniently makes this data available via their REST API.  All products discovered in this way are set to unapproved status pending manual review.

Next, I set up another scheduled task to populate and refresh metadata about the products via Amazon’s Product Advertising API.  Per Amazon’s terms in order to display price information, this data has to be refreshed hourly.  For efficiency I request data on batches of ten products at a time, which is the maximum limit.

I created a manual process for reviewing and approving products to be shown.  This process includes writing a custom description, adding relevant tags (e.g. “For Kids” or “Gadget Lovers”), and setting an age range and gender specificity (if applicable).

The UI is written in JSP and outputs HTML5.  Some features are powered by javascript, such as the price history button which uses flot to render the graph of historical price data.

Spring 3.1 ties it all together.  Spring MVC handles the front-end.  Spring JDBC is used for interacting with PostgreSQL.  I could have used Spring’s event system, but I wanted to get some experience with ActiveMQ.  There are a number of message senders and listeners set up for events such as “price changed”  or “product suggested”.

I’ll probably think of a snappier name eventually, but for now I registered http://spectacular.gift (new “.gift” TLD).  Have a look if you like!  It’s basically in beta, and I’m still adding new products and tags.

25
Feb

Web Page Optimization – Changes I Made for Page Loading Speed

It has been years since I optimized the page loading speed at GoToQuiz. At that time, image sprites and CSS/javascript minification were the state of the art. In the intervening years, though, further progress has been made. Of particular note is the “retirement” of older versions of Internet Explorer, and the emergence of CSS3 and HTML5, which allows developers the ability to further streamline their sites. So as I am in the process of revamping the UI, I’m taking advantage of the latest techniques to boost page loading speed. Be aware, this is nothing revolutionary–this blog post is simply an overview of which changes I’m making, and why.

Migrating pages over to HTML5 allows the markup to be more semantic while taking fewer bytes–a double win.  Using tags such as nav, section and article helps alleviate so-called “div-itis”, allowing cleaner markup and a smaller overall filesize.  HTML5 also follows the maxim of “convention over configuration”, letting you eliminate unnecessary attributes like type=”text/javascript” on script tags, for example.

The adoption of CSS3 has provided many possibilities for creating beautiful visual effects without using image files.  Here is an example: on an early iteration of the design of the various headings on GoToQuiz, I used this sprite combined with CSS to create the rounded color bar appearance.

Read moreRead more

9
Feb

Simple, Light-Weight Radio Button Replacement jQuery Plugin

I was looking for a jQuery plugin to replace the standard browser radio buttons with custom images, but I had a few requirements. 1) Tabbing and focus had to function correctly, and 2) the javascript had to be light-weight. After looking into some heavy plugins and some simple ones that didn’t handle tabbing/focus properly, I decided to roll my own.

Here’s the plugin code, with comments:

(function($) {
    // source: http://www.gotoquiz.com/web-coding/
    'use strict';
    
    //the plugin
    $.fn.niceRadio = function(options) {
        if (this.length !== 0) {
            //initialize settings & vars
            var settings = $.extend(
                {   baseClass:      'nr_base',
                    focusClass:     'nr_focus',
                    checkedClass:   'nr_checked'},
                options), 
                $radios = this,
                $aset = $();
            
            $radios.each(function(i, el) {
                var $radio = $(el),
                    //create an anchor per radio that will be our radio replacement
                    $a = $('<a href="#" id="nr_a'+ i +'" data-nr-id="'+ i +'" class="'+ 
                            settings.baseClass +' nr_for_'+ $radio.attr('name') +
                            '" tabindex="-1"></a>');

                //give each radio a custom class with a counter (nr_r#) for easy lookup
                $radio.addClass('nr_r'+ i)
                    //store the counter value as data as well
                    .data('nrId', i)
                    //insert the anchor immediately after the radio
                    .after($a)
                    //hide the radio visually
                    .css({position: 'absolute', 'margin-left': -2999});
            
                //if radio was already checked, add the checked class to it's replacement
                if ($radio.is(':checked'))
                    $a.addClass(settings.checkedClass);

                //collect the set of anchors as they are created
                $aset = $aset.add($a);
            });


            //when an anchor is clicked...
            $aset.click(function(e) {
                e.preventDefault();

                //$clicked is the anchor
                var $clicked = $(this),
                    //$checked is the radio to be checked
                    $checked = $('.nr_r'+ $clicked.data('nrId'));

                //check it
                $checked.prop('checked', true)
                    //focus it
                    .focus()
                    //trigger the change event on it
                    .change();
            });


            //when a radio received focus, add the focus class to its replacement
            $radios.focus(function() {
                $('#nr_a'+ $(this).data('nrId')).addClass(settings.focusClass);
            })
            //when a radio loses focus, remove the focus class from it's replacement
            .blur(function() {
                $('#nr_a'+ $(this).data('nrId')).removeClass(settings.focusClass);
            })
            //when a radio is changed, call the changeEvent function
            .change(function() {
                changeEvent($(this));
            });


            return $radios;
        }

        //called when a radio selection changes
        function changeEvent($radio) {
            //if $radio is checked...
            if ($radio.prop('checked')) {
                //remove checked class from any other anchor associated with
                //radios of the same name
                $('.nr_for_'+ $radio.attr('name')).removeClass(settings.checkedClass);
                //add checked class to this radio's replacement
                $('#nr_a'+ $radio.data('nrId')).addClass(settings.checkedClass);
            }
            else
                $('#nr_a'+ $radio.data('nrId')).removeClass(settings.checkedClass);
        }
    };
    
})(jQuery);

This requires some CSS rules, which might look like:

.nr_base		     { display: inline-block; 
                       height: 16px; 
                       width: 16px; 
                       background: transparent url(radio_sprite.png) no-repeat 0 0;
                       vertical-align: middle; }
.nr_checked	         { background-position: -16px 0; }
.nr_focus		     { outline: 1px dotted #555; }
.nr_checked.nr_focus { outline: none; }

Minified, this radio replacement plugin shrinks to under 1KB. It handles tabbing and focus properly, for both screen readers and keyboard users. It handles radio changes via clicking associated labels. And by using an anchor tag to hold the radio replacement image, it handles the odd browser that does not treat labels as clickable (a problem with some of the other radio replacement plugins I encountered).

Feel free to use this plugin as you like, and I hope it proves useful! Let me know if you find any bugs or particular browser quirks.

21
Aug

Log SQL Statements With Parameter Values Filled In (Spring JDBC)

If you’re using parameterized queries with Spring JDBC (and why wouldn’t you?), it’s easy to log them with a simple log.debug() statement. But your logged statements will be full of ?’s instead of the values, which makes them much less useful. Suppose you’d like to know what was substituted for those question marks. It’s not so difficult if you use AspectJ.  Here is what is needed:

  1. aspectjrt.jar and aspectjweaver.jar from here.
  2. An aspect with a pointcut on the JdbcOperations interface, which JdbcTemplate implements.
  3. @Before advice that intercepts the execution of JdbcTemplate methods and logs de-parameterized SQL statements.
  4. Configuration of Spring applicationContext.xml to get it working.

Let’s start off with our class:

@Aspect
public class SqlLogger {
    private static final Logger log = LoggerFactory.getLogger(SqlLogger.class);

Here, I’m using an org.slf4j.Logger, but any logging framework will work.

Next step is to add the method that will capture the SQL and parameters as they are executed. Again, I’m using Spring JDBC, so all calls are made to an object that implements JdbcOperations.  We can set up our class to spy on all relevant calls by defining our method like so:

Read moreRead more

2
Aug

Make Rainbow-Colored Lists (and other HTML elements) – jQuery plugin

How to make rainbow-colored lists? I started searching for a solution and found a plugin for rainbow text, which inspired me to write my own jQuery plugin to rainbow-ify lists and other sets of HTML elements.

I’m used to working with RGB hex color codes.  But there is no intuitive way to generate a spectrum of color using RGB, so I turned to HSL (hue, saturation and lightness).  Most modern browsers support colors specified with hsl(hue, saturation%, lightness%), although IE versions prior to 9 do not. So I started out using HSL, adding IE support later. I found that keeping the saturation and lightness constant at 100% and 80% respectively while increasing the hue in steps produces a really nice rainbow effect.

Let’s back up a bit. Consider the following styled list:

rainbow list w/o rainbow!

I’d like to color this list to look like a rainbow. Using CSS alone is unwieldy; without knowing in advance the number of elements in the list, it’s impossible.  We need to use Javascript to get this result:

rainbow-colored list

Here’s how I did it–my rainbow plugin code (~1KB when minified):

Read moreRead more

16
Mar

jQuery Shake Plugin to, er, Make Elements Shake

I wanted to make a submit button shake back and forth when the form input is invalid, signaling the user that there was a problem. Simple enough. So I googled around for an existing solution, but the jquery plugins I found only made elements vibrate like crazy. I wanted a left-right shake.

Well it turned out to be very easy to implement myself.  Minified, my shake jquery plugin is less than 0.5KB.  Here’s a demo page.  Here’s the code:

(function($) {
    'use strict';

    //the plugin call
    $.shake = $.fn.shake = function(options) {
        if (this.length !== 0) {
            //initialize settings & vars
            var settings = $.extend(
                {   distance:   12,
                    iterations: 3,
                    duration:   160,
                    easing:     'swing'},
                options),
                $shakeThis = this;

            return $shakeThis.each(function(i, el) {
                var $el = $(el),
                    marg = ($el.parent().width() / 2) - ($el.outerWidth() / 2);

                $el.css('margin-left', marg);
                for (var j = 0; j < settings.iterations; j++)
                    leftRight($el, marg);

                $el.animate({'margin-left': marg}, settings);
            });
        }

        function leftRight($el, marg) {
            $el.animate({'margin-left': (marg - settings.distance)}, settings)
               .animate({'margin-left': (marg + settings.distance)}, settings);
        }
    };

})(jQuery);

Read moreRead more

30
Sep

Distribute Android App From Your Own Web Server

Listing your app on Google Play or another centralized app store may not make sense for your situation.  This was the case for me when I created an app for small web community, which would be meaningless for anyone else.  There’s nothing stopping you from simply uploading the .apk file to your web server and linking to it, though.  And that’s what I did.

One thing I discovered is that not all devices will recognize the file as an installable app based on the file extension alone.  You must ensure you send the proper mime type.  You can achieve this for Apache by associating the correct mime type with .apk files.  Adding this line to .htaccess will do the trick:

AddType application/vnd.android.package-archive apk

When a user clicks a link to the file, their device will download it.  There will be a “download complete” notification when completed.  Clicking this will grant the option to install the app.  It’s a process that is likely to be unfamiliar to your users, but it’s not too complicated.

One thing you lose is the automatic management of app updates that Google Play provides.  Your users will have to manually download the latest version of the app and install again, which thankfully is handled properly at that point by Android–that is to say, it’s treated as an upgrade to a newer version rather than a clean install.  Consider adding some type of notification to the app’s functionality that will alert users to perform an update if their app version is not the latest.

29
Sep

How To Make an Android Home Screen Widget Update Only When Visible

Depending on the nature of your home screen widget, you may want it to stay closely synched to real-time data. It would be nice if Android fired a system event that you could register for whenever your widget is visible on the screen. However, Android does not do this, and could not be counted upon to do so in any case because some users may be using a 3rd party alternative to the default home screen.

You’d like to update your widget early and often, at times when the user might be looking at it. But you want to prevent it from updating (and wasting resources) when the user is not engaged. There is a way to achieve this. This technique isn’t perfect, because it will cause your widget to update even when hidden behind another running app, but that much is unavoidable. (Note to Google: an isWidgetVisible() method would be great!)

I’m going to describe how I used a Service to update my web community widget only when the screen is on.  The java classes involved are:

  • MRPWidget (extends AppWidgetProvider)
  • WidgetUpdate (extends IntentService)
  • MRPAlarmReceiver (extends BroadcastReceiver)

I described the MRPWidget class here: creating an Android home screen widget.  Now I will describe how the Service is used to update the widget.

Read moreRead more

28
Sep

Android Home Screen Widget: Creating the AppWidgetProvider

Extending AppWidgetProvider is a simple way to encapsulate your app’s management of your home screen widget.  It provides several widget life cycle methods you can override to handle such events as a user adding or removing the widget from their home screen.  It also provides an onUpdate method that ties into the periodic updates the Android system will trigger for your widget.  For my purposes, I disabled this periodic updating because I wanted to control it myself via a Service.

Your AppWidgetProvider interacts with the widget via the RemoteViews class.  The widget is itself hosted in a separate process from the one in which your app runs, and RemoteViews provides methods to cross that gap, update the views and add click listeners.  Note that RemoteViews methods provide only a limited subset of view functionality, and this may restrict what you’d like to do with your widget.  For example, animations are not supported.

I’m going to describe how I created my web community widget.  The java classes involved are:

  • MRPWidget (extends AppWidgetProvider)
  • WidgetUpdate (extends IntentService)
  • MRPAlarmReceiver (extends BroadcastReceiver)

My widget class is named MRPWidget, and it extends AppWidgetProvider.  All visual updates to the widget are handled by this class, as well as starting the service that will perform the updates.  For example, when the widget is enabled it should start the service: Read moreRead more

27
Sep

Creating an Android Home Screen Widget

The home screen of a smartphone is precious real estate.  As users interact with their phone throughout the day, the icons and widgets on the home screen are constantly seen and ready to be interacted with at a moment’s notice.  Offering a widget for your app, service or web site is therefore a uniquely compelling way to to boost its visibility and “stickiness.”

One site that I’ve built is a small online community where people interact mainly via message boards and private messages.  I had already built a simple Android app that users could install to notify them whenever there is a new reply or message.  But to take this app to the next level, I needed to add a home screen widget.

This widget would stay updated with the most recent post made on the site as well as show the names of the users currently active.  In this way, the widget would effectively be a small window into what was going on at the site at any given moment.  Clicking the widget should take you directly to that latest post, logging you in automatically.  Finally, the widget should also visually alert you to any pending unread messages.  It should do all this while taking up as small a footprint as possible, as smartphone users are reluctant to give up too much of their screen to a single widget.

So there are a number of challenges, each of which must be resolved.  Which specific data must be displayed on the widget (we want as much as possible without forcing a larger footprint)?  How to retrieve that data from the web site?  How to keep the data fresh while minimizing battery and data usage?  How to take the user to the correct page when the widget is clicked?  How to make the widget visually appealing?  This last one is difficult, speaking as someone who is not a graphic designer, but I gave it a solid try. 🙂

So in the next few posts I’ll go into how I created an Android widget. But first please have a look at it:

Screen cap of Android Home Screen Widget

The footprint is 2×1 cells on the home screen.  The top half displays the latest post on the site, alongside the user’s icon, with the user’s name (“Example User”) followed by the message board it was posted to.  The bottom half lists the names of currently-active users. Had there been more than can fit on one line, the text here would scroll as a marquee.  Not shown is a hidden middle section that expands to notify of unread post replies and private messages.  All of this is displayed atop a semi-transparent 9-patch image that stretches to encompass the content.

More details on crafting this widget to follow.