Personalization, A/B Testing, Flicker and Convenience

All images created with Adobe Firefly

I recently read Pedro Monjo’s article on “Flicker, personalization and page speed” [1]. His hypothesis — in short — is that when you want to add personalization to a page with a tool like Adobe Target, there are three major conflicting properties:

• Fast page loading (and rendering)
• Personalization
• No flicker

Pedro postulates, you can only have two of the above properties — but not all three.

The issue lies in the nature how personalization tools work.

In short:

A page is loaded in the browser, the personalization engine kicks in and changes parts of the pages DOM according to the segment a user belongs to.

These changes result in visual “flicker”. Parts of the contents are changed after loading — while the user already interacts with the page. To avoid this, a personalization engine would hide the whole page until all changes are complete. The avoids visual changes — but obviously defers the visual rendering of a page.

Pedro then continues this could be avoided by doing the personalization server-side, which in practice, nobody does, because this requires more server-side resources than would be economically feasible.

Pedro’s conclusion: In “real-life”, personalization you must be willing to make compromises and sacrifice one of the desired properties.

Convenience

While I agree with Pedro’s opinion in general, I like to add a fifth dimension which can open opportunities for other solutions:

• Fast page loading and rendering
• Personalization
• No flicker
• Server resources
• Convenience

Upon closer inspection we see that the problem is how the personalization tools work: They are deployed and managed independently from the rendering engine (e.g. CMS or eCommerce system). The rendering engine is not aware of the personalization efforts. The integration happens at DOM level and as such is very generic.

This is very convenient and budget efficient as it applies to a broad range of use cases. But it comes with a cost: You sacrifice performance.

My hypothesis is, by closer integrating the systems, and some customizations some of the negative effects can be mitigated.

Don’t get me wrong: Pedro’s “trade-off triangle” is still valid: We’ll solve the issues by putting in more resources (aka “budget). Only now in the form of custom implementation and content management efforts instead of more servers.

And we will make compromises: Only now we can do so more fine-grained instead of all-or-nothing.

Example: AEM and Adobe Target

For illustration purposes, I’ll assume, the rendering engine is Adobe Experience Manager (AEM), and the personalization engine is Adobe Target (Target). But the principles are similar for other combinations of tools.

Why page speed matters

A quick recap, why page speed is so important.

a) Page speed — in part — determines how a page ranks in search engines. A better ranking means more traffic and more potential conversion and thus more revenue.

b) Slower pages bear the risk that potential customers lose patience and leave without converting.

So, it’s — again — all about money. But here is the thing: You can balance how much you want to invest in implementation or to counter lost opportunity costs.

Adobe Target and the Visual Experience Composer

Nowadays, Adobe Target users favor using the Visual Experience Composer (VEC) to manipulate the DOM in A/B tests and personalization campaigns. VEC provides a convenient way to manipulate the whole page. But: Because the whole page can be potentially manipulated, it is necessary to also hide the whole page during manipulation to avoid flicker which is perceived as slow and non-responsive.

A/B Testing vs. Personalization

What we need to understand is, there are fundamental difference between A/B tests and personalization.

A/B tests run for a limited amount of time to test a hypothesis how a website should be globally changed to improve overall conversion. As a simple example, think of an A/B test to prove that a larger shopping basket in a more prominent position on the page leads to more checkout activities. This improvement applies for all users. I.e., I can’t imagine that one segment of users expects the cart in the bottom left, while others in the top right. That’s why insights gained during an A/B test result in a global change on the site — that than is implemented in AEM.

Personalization activities typically run for a longer time: For the duration of the planned personalization campaign — or even indefinitely. These activities often take place on a pre-defined space of the page. Again - it’s unlikely that you would target different segments with different layouts. It’s more likely that you target segments with specific content. Think for example of a personalized ad campaign in the hero banner above the fold.

These differences are often diffused because nowadays the Target community seems to favor VEC [2] for both use cases.

For A/B tests, VEC is an excellent choice. It provides flexibility for A/B test in a broad range of hypothesis while at the same time the UI is convenient to use. As mentioned earlier, the convenience comes with a price: Potential flicker, that must be suppressed by hiding the page during manipulation. This is why the insights of the A/B should be implemented in the rendering engine — to speed up the perceived page load.

Personalization activities usually do not require the same level of flexibility. VEC is not the only choice here. One could also use local mboxes and form-based activities [3]. This in turn means, that you must prepare the webpages to mark local mboxes be able to address them in Target. To be more precise, this does require some up-front implementation in AEM to add mbox markers at component level. This pays off eventually, though:

a) It reduces or avoids flicker and

b) can be used by less tech-savvy marketeers.

(I’d argue, that VEC is too powerful to give it into the hands of non-tech marketeers.)

On the downside, personalization activities now need to be prepared in both AEM and Target, which might be perceived as … well … less convenient. That’s the trade-off.

Mitigate adverse effects

The remainder of this article is a loose collection of ideas about avoiding or reducing adverse effects like slow page rendering, SEO downranking or flicker. Neither of which is a silver bullet, but using some with good measure can reduce certain effects. You can balance between implementation effort and effectiveness.

Avoid “all or nothing”

Reduce range of pages

Only apply flicker suppression on pages that are part of an actual A/B test. If a page is not manipulated in a test, you do not have to hide/show its contents. Mark a page (or subsite) partaking in a test in AEM and only hide/show the contents of those pages. This reduces the number of pages with a potential lower conversion rate (due to presumed lower rendering performance).

Limit flicker suppression to active test periods

This is the same argument: Only apply flicker suppression while a test is running — not per se. Remove it after the test is complete. This reduces the time in which you potentially lose on a performance-related lower conversion rate. There is also a chance, that search engines do not re-rank during the test or that the page rank recovers after the test.

Limit test sample group

If you have massive amounts of traffic, you can think of applying the A/B test to only a portion of the users. E.g. you could use SSI or ESI (Server Side Includes, Edge Side Includes) to run the test only for odd IP-addresses. Thus 50% of the users still convert normally. The other 50% would be subject to A/B testing, further subdivided into test and control group. Admittedly, this means that the test needs to run longer to reach the same level of significance. The same effect can be reached by limiting the testing period. I am just thinking loud here to spark your own creativity ;-)

Placeholders and lazy loading

We are all used to large images loading a bit slower than the page itself and to appear with a small delay. So, it would not be unusual to have a hero ad appear a bit later than the rest of the page. To avoid flicker or content shift, provide an empty placeholder instead of the hero ad. This requires that you control the size (i.e. height) of the ad to avoid content shifting. Also, it requires you to implement the fallback ad in AEM. In case the call to Target times out, you would display an ad that you have as a “reserve” statically loaded in a hidden section of the page.

Off-screen manipulation

When the personalized area is below the fold, flicker is not an issue. The manipulation happens off-screen, and you don’t have to worry about flicker. Of course, everything below the fold and requiring scrolling is not as effective as above. Arguably, targeting below the fold is not worth the effort. A basic rule of targeting is: Be bold! Do not invest in subtle changes or differences.

Carousels

Changes can be both off-screen and above the fold. E.g. when you have a carousel on the hero position of the page, the first pane can show a generic ad while loading the personalized ad(s) for the subsequent panes in the background. Rolling through the adds is an intentional manipulation of the visible DOM and masks flicker.

Interactivity

Another way to “sneak in” personalization is by providing a generic teaser that is converted into a personalized offer upon click. E.g. a teaser showing a wrapped gift with a text “Your personal Christmas gift: 10% off”, which then is replaced by the actual offer when the user clicks. This approach can engage users by appealing to their natural curiosity and mask flicker.

Sometimes, limitations can work to your advantage. Be playful and creative.

Recommendations

Recommendations (“You might also like…”) typically are below the fold. You don’t have to worry about flicker or shift at the bottom of a page.

Embrace flicker: Splash ads

Another way to personalize is to display a splash ad in a popup on top of the actual page. This causes the same visual distraction as flicker does. But here, this is used as a feature. You want to catch attention. The good part: You don’t have to worry about flicker suppression. Also, you don’t have to worry about fallbacks in case of timeouts. If Target times out — just don’t display the splash ad.

You need to be careful with splash ads, though. They can become annoying if overused. Only use them on special occasions.

AEM implementation hints

Most of the ideas above require some custom implementation in both AEM and Target. There is never a free lunch. You must decide how much you want to invest in your own framework. The more you plan to use personalized hero banners or splash ads the more it makes sense to invest into a user-friendly solution.

My advice: Start with “out of the box” Target — sacrificing some page performance — and if you feel that your personalization campaigns show effect, plan a more robust solution that mitigates some of the undesired effects.

Here are some tips for implementing personalization with Target in AEM.

Separate content and configuration

Earlier I proposed to “mark” pages subject to testing in AEM. However, we do not want to do that in the page properties.

a) This would require that you re-publish the page to take effect — which you can’t if it is currently being reworked on the Author, and you can’t re-publish this work in progress just now.

b) This would require marking and publishing multiple pages when the test is supposed to be site wide.

Instead, create a configuration page to list all pages subject to A/B testing (or path patterns) and publish only this configuration page. If a normal content page renders on Publish, it checks if it is listed in the configuration. If so, hide the html body to suppress flicker.

Use servlet filters to add local mbox markup

AEM — in contrast to standard JEE — provides servlet filters at component level [4]. To markup a component as a local mbox you do not have to do that for each component, but as a global aspect.

• Provide a dialog tab to capture the id and default height of the component.
• Include that dialog tab on each component you want to manipulate.
• Create a component-level servlet filter that checks if the component has an mbox id (set by above dialog tab) and if so, wrap it in a div that marks the boundaries of the mbox (and set the default height to avoid content shift).
• Create global JS code to load the local mboxes and handle timeout cases.
• Upload Experience Fragments from AEM to Target to be used as “offers”
• In Target, create a form-based activities to replace the defined mboxes with Experience Fragments.

Conclusion

I recently read about a story where — back in the days - a social media service was uploading profile pictures during the registration process before the actual form was submitted. If the form was not submitted, the picture was discarded eventually. But if it was submitted, people were amazed how “fast” the service was.

The bottom line is: Sometimes some trickery to mask or embrace issues can be more effective than trying to brute-force a solution.

With a bit creativity and out-of-the-box thinking one can produce smart solutions.

I hope I have sparked a bit of your coding creativity in this article.

Happy targeting

-achim

References

[1] Flicker, personalization and page speed

[2] Visual Experience Composer

[3] Form-Based Experience Composer

[1] Sling Servlet Filter Support