Child pages
  • 1. Tracking Events
Skip to end of metadata
Go to start of metadata

The Recommender System collects information from the web presence of the customer to generate personalized or context based recommendations for a user profile. An event tracking process collects events. Most popular examples of such events are

  • "Click", if a user opens a product page
  • "Buy" or "Consume", if the user buys a product or consumes in some other way (watches video, or reads an article)
  • "Rate" if a user likes, comments or rates a product
  • "Login" if a user logs in on a website

See user guide chapter  8. Event Types for a brief introduction to different types of events. Depending on the event type, some additional parameters like product price or user rating must be provided.

 

Some notes about item identifiers

Keep in mind that the decision what to track is crucial for the whole recommendation process. As a customer or more precisely the specialty department you need to evaluate what should be presented as recommendations on a site. In the eCommerce business you mostly have the possibility to track items based on:

  • Stock Keeping Unit (SKU) or
  • Master Item Identifier (MII)

Exactly the identifiers that are tracked will also be recommended (what-you-track-is-what-you-get). By default it is not possible to track SKUs but recommend MIIs. Following is an example of a typical decision case in the eCommerce business:

Customer A implements the YOOCHOOSE Recommender Engine and decides to use the SKU as item identifiers and recommendable items. What could happen, is that users browsing through the shop will probably get recommendations of the same item which is currently displayed but in a different size and/or color, a so called "variant". As the Recommender Engine doesn't know anything about the relations of items, every single SKU is used to calculate similarites between items. In the case of bestsellers, this could lead to the appearance of a shirt in size L on position 2 and the same shirt in size M on position 4.

Customer B decides to use the MII as item identifiers. This will lead to recommendations that will not contain variations of the currently shown item. In this case it could not happen, that on the detail page of shirt X the same shirt in a different size is recommended. In the case of bestsellers it is therefore not possible to have the same shirt on two positions in the list of recommendations.

If the size of a product or the color is selectable on a detail page of an item, you may prefer to use the MII. If recommendations of the same item in different sizes or colors are desired, you should use the SKU as item identifiers.

Keep in mind to use the same item identifiers also in the check-out process if a user buys an item.

 

Implementation

An event (for example "click") must be reported to the Event Tracker calling a predefined URL. For example:

http://event.yoochoose.net/api/10000/click/max.musterman/1/100

The most simple way to do it, is placing a tiny fictive image on the product page like one do it for integrating an analytic service or visitors counter.

Reporting other events (like "Buy" or "Rate") is more sophisticated. The event call can be done on the server side, using JSONP or using combination of server side scripting and Java Script.

User Tracking

High quality recommendations can only be delivered if the underlying data is correct and consistent. For a consistent tracking it is crucial to use the correct identifiers for a user and send the events accordingly to the actioins of a user on a website. Usually a user visits a website as an anonymous user. Therefore its identifier is either an own, first party cookie or the session ID provided by the webserver. If there's no login mechanism we encourage to use an own cookie and set the expiry date to at least 30 days in advance from the last usage. If there's a login mechanism, the user is tracked with a temporal identifier before the login. Immediately after a successful login process a "login" event (see below) must be sent and from then on the pseudonymous user identifier (e.g. a system's internal registration id) must be continously used. After (automatic) logout, the anonymous user id should be used again without sending any "logout" event. This event is not needed and therefore does not exist.

The user identifier is required in the tracking request, otherwise it is discarded from the tracking servers.

If you use cookies, please verify that you put some "dummy" value as identifier in the tracking request in order not to loose the tracking information if the browser has JavaScript or cookies disabled. Even if the event is not user specific it is still useful for popularity models.

Click Event

This event must be reported to the Recommendation System, if a user opens a product/article page or any other page with detailed content related to a single item or a list of items. Click events also deliver information about the structure of the web presence.

We do not store the user ID itself in the database. All user IDs are non-reversible anonymized before the user profile is stored to the disk or the recommendation models are built.

The URL to track click-evens has following format:

http://event.yoochoose.net/api/[customerid]/click/[userid]/[itemtypeid]/[itemid]

Name

Description

Values

customerid

Your customer ID, a.k.a. "mandator ID".
(for example "00000").

alphanumeric 

userid

The ID of the user in the web presence of the customer. It could be internal customer code or session code for anonymous users. See chapter "Request Format and Parameters" bellow in this document for more information to the user ID.

URL-encoded alphanumeric

itemtypeid

Item type ID. See the chapter Item Type ID for more information. See chapter "Request Format and Parameters" bellow in this document for more information to the item type ID.

1 – 5

itemid

Unique ID of the item the user has clicked.

Item IDs based on string values

We also support string identifiers for item IDs to track content on a website but it is discouraged due to fraud and security issues. If you are not able to provide numeric identifiers for the tracking process, please get in touch with us under support@yoochoose.com for further information and implementation notes.

1 to 2147483647

All embedded parameters are required for the request. Some optional request parameters can be set over query string parameters (GET parameters).

http://event.yoochoose.net/api/[customerid]/click/[userid]/[itemtypeid]/[itemid]?parameter1=value1&parameter2=value2  

Name

Description

Values

categorypath

The forward slash separated path of categories of the item. As all other parameters it must be URL-encoded, for example
%2FCameras%26Foto%2FCompact%20Cameras%2FCanon 

See chapter 7. Category Filter of the user guide for additional information to use cases the category path is used for.

URL-encoded string.

Forward and taling slash is ignored "/Cameras/" is the same as "Cameras"

Clickrecommended Event (a.k.a. Follow Event)

If a user clicks on a recommended item the following event should be sent to gather statistics over the acceptance of recommended items. Click-recommended events are also known as follow-events. The URL has the following format:

http://event.yoochoose.net/api/[customerid]/clickrecommended/[userid]/[itemtypeid]/[itemid]?scenario=<scenarioid>

All embedded parameters are the same like for a click event. The request parameters are:

Name

Description

Values

scenario (required)

Name of the scenario, where recommendations originated from.

URL-encoded alphanumeric

The scenario parameter identifies the originating scenario to gain detailed statistics about which scenario motivated the user to click a recommendation. This information comes with the recommendation from the recommendation controller. See chapter 2a. Fetching Recommendations (deprecated) for more information.

This event is used for providing statistics about how often recommendations of the configured recommendation scenario wer accepted or considered as useful by the users. Besides that this information is used for providing and evaluating A/B Tests.

Consume Event

Publisher vs. Shop solution

The "consume" event is important for publisher/media web pages (like a newspaper, news agency or online movie provider). For web shops this event in general not needed but can of course be used by own implementations.

The event should be sent if the user stays on the page for some predefined period of time. In this case it is assumed, that the user consumed the item (read an article or finished watching a video).

The URL has the following format:

http://event.yoochoose.net/api/[customerid]/consume/[userid]/[itemtypeid]/[itemid]

All embedded parameters are the same like for a click event. The request parameters are:

Name

Description

Values

percentage

It defines how much of an item was consumed, e.g. an article was read only by 20%, a movie was watched by 90% or someone fineshed 3/4 of all levels of a game

0-100

The logic how to calculate the percentage is to be defined by the implementation. For articles this could be by scrolling events or for a movie based on the consumption part. Keep in mind that you should put some brainwork what 100% consumption means. For example a movie contains a closing which is probably almost never consumed and therefore should not be part of the percentage calculation.

 The simplest implementation for articles is a JavaScript timer, which sends this event after some predefined time is elapsed (and the user did not leave the page). The timespan after that the event is triggered, should be dependent on the content. E.g. it could be 30 seconds for a small newspaper article or a timespan calculated by the amount of words.

Incremental tracking of consume events

Consume events for a user can be sent incrementally as the recommender uses only the highest percentage rate. For example if a user watches a movie, the site could sent consume events in 10% steps in order not to lose tracking information by forcibly closing the browser window.

Buy Event

As the name suggests this event must be used if the user buys an item. It must be sent to the event tracker at the end of the check-out process of a shopping basket to ensure that no further action of the user can result in an abort.

The URL has the following format: 

http://event.yoochoose.net/api/[customerid]/buy/[userid]/[itemtypeid]/[itemid]?fullprice=2.50EUR&quantity=4

Additionally to the fact of buying, this event should provide information about the product price and quantity.

Name

Description

Values

quantity (required)

The number of items the user bought. Default value is "1". It is possible to send n events instead of setting this parameter to n.

Integer
(default is "1")

fullprice (required)

A price for a single item. It contains the price in decimal format plus the currency ISO 4217 code. If the price has a decimal part, the point must be used. No space must be between price and currency.

For example "12.45EUR" or "456JPY "

price
(deprecated) 

This parameter is deprecated, use the "fullprice" instead!

The price for a single item in Euro cents, e.g. 2.25EUR is price=225, default currency "EUR" is assumed.

Integer
currency
(deprecated) 

This parameter is deprecated, use the "fullprice" instead!

The currency of a price. If left out the default currency "EUR" is assumed.

ISO 4217

For example, if a user bought 4 pens for 10 Euros, the parameter "fullprice" must be set to "2.50EUR" and parameter "quantity" to 4.

The "buy" event is only relevant, if the user is charged per item like it happens in a classical shop. If items are sold on subscription base or the web presence is ad-sponsored, this event type is not needed.

Prices in a Buy Event

As mentioned above every buy event can contain a price. If the price is set, it is stored with the event and used for calculating revenue for statistics. The price should be a price the user paid for the item including all taxes and discounts. The price provided in the buy event is used only for building statistics.

If item price filtering is activated the information provided over the item import is used. See chapter 3a. Data Import, Items for more information.

The currency is stored with the price and normalized only when statistic information is requested. It is often a good option to select a base currency and convert prices before sending the buy event. The price attached to a buy event does never overwrite the price which was defined in an import.

Login Event (a.k.a. Transfer event)

Recommendations rely on the fact that user actions can be correlated over a longer period of time. Moreover, recommendations like "users who viewed this item ultimately bought it" require correlating click events with subsequent purchase events. In general, users tend to browse a site anonymously and add items to their shopping cart. Up to this point, the user is identified by a visit-scoped variable (e.g. a session ID or a first party cookie). During check-out of the shopping cart, a user probably logs in to an existing account. As  consequence, the user identifier changes from an anonymous visit-scoped ID (sourceuserid) to a pseudonymous, persistent account ID (targetuserid). It is highly desirable to correlate both IDs in order to correlate the buy events (account ID) with the preceding click events (visit-scoped ID). The administrative event 'login' serves exactly this purpose.

The format of the URL is: 

http://event.yoochoose.net/api/[customerid]/login/[sourceuserid]/[targetuserid]

Name

Description

Values

sourceuserid

User identifier valid up to now(usually some anonymous session ID)

URL-encoded alphanumeric

targetuserid

User identifier valid from now on
(usually an account ID or login name)

URL-encoded alphanumeric

Deprecated 'transfer' event

Based on feedback of our customers and to make the purpose a bit clearer we renamed the formerly named 'transfer' event to 'login'. As the name implies it should be used when a user logs in. It covers exactly the same functionality like the transfer event. There's no need to change anything on the customer's side for existing implementations as the transfer event http://event.yoochoose.net/[solutionid]/[customerid]/transfer/[sourceuserid]/[targetuserid] will still be supported but should be considered as deprecated!

Basket Event

The basket event can be used to add items to a user's shopping cart. This event is especially useful if anonymized checkout is allowed or there's no recurring user identification possible. By using the shopping cart items as input for getting recommendations, the empty profile or cold-start problem (meaning the user has no buy history) can be solved. The more valuable basket events instead of recent user clicks can be used to provide personalized recommendations. It also happens quite often that users "store" items on their shopping wishlist and plan to buy them later. With the help of this information, personalized shopping cart based recommendations can be provided in the whole shop.

http://event.yoochoose.net/api/[customerid]/basket/[userid]/[itemtypeid]/[itemid]

 

Name

Description

Values

categorypath

The categorypath from where the item is placed into the shopping cart.

URL-encoded string

The categorypath parameter should be added due to statistical reasons. Often items are multihomed in a shop or a media/publisher site. In order to analyze from which location these items were sold most, this value should be provided.

Rendered Event

Render events must be sent, if the web presence uses the recommendation provided by the recommendation engine and renders it on the web-page. In a combination with a predefined threshold it allows the recommender engine to exclude this item from a future result and not recommending the same item to the same user multiple times during a session.

The URL for a render event has the following format:

http://event.yoochoose.net/api/[customerid]/rendered/[userid]/[itemtypeid]/[itemid[,itemid]]

The render event has the same embedded parameters as a click event except of the item ID. It is common that recommendations are rendered as a block with multiple items. To save traffic and reduce latency, it is possible to bundle multiple recommendations in one request. Several item IDs must be comma separated.

Rate Event

Publisher, media or shop portals often allow commenting products, articles or movies.. If a user comments an item it indicates a special interest on this topic that has to be treated separately.

The format of the URL is:

http://event.yoochoose.net/api/[customerid]/rate/[userid]/[itemtypeid]/[itemid]?rating=50

This can also be used for explicit ratings like a five star rating or any other liquor scale based rating. The predefined rating can be submitted when the user comments on an item.  

Name

Description

Values

rating

The rating a user gives an item

Integer from 0 to 100

Scale of rating values

The rating value is normalized during the caculation of rate based recommendations. So there's no need to use the full scale of 0-100 but it needs to be consistent.

Blacklist Event

If the web-page offers a link or button that allows feedback like "do not recommend this item to me anymore" the user could express that he might bought this item on behalf of somebody else or changed his opinion after reading.

The format of the URL is:

http://event.yoochoose.net/api/[customerid]/blacklist/[userid]/[itemtypeid]/[itemid]

There are no query string parameters for this event. 

Examples

The user "Js79009234YU7" navigates to an item with the id 123 and type "1" which is located under Shoes -> Children:

http://event.yoochoose.net/api/00000/click/Js79009234YU7/1/123?categorypath=%2FShoes%2FChildren

Products 128, 129 and 155 of type 1 are rendered as recommendations for user "Js79009234YU7". Recommendations were delivered by the scenario "also_bought":

http://event.yoochoose.net/api/00000/rendered/Js79009234YU7/1/128,129,155

He clicks on recommended product 155 that was delivered by the call of scenario "also_bought":

http://event.yoochoose.net/api/00000/clickrecommended/Js79009234YU7/1/155?scenario=also_bought

User "Js79009234YU7" has watched a video with the id 452 (videos have item type "3"):

http://event.yoochoose.net/api/00000/consume/Js79009234YU7/3/452

User "Js79009234YU7" has watched a video with the id 466 by 60 percent (videos have item type "3"):

http://event.yoochoose.net/api/00000/consume/Js79009234YU7/3/452?percentage=60

User "Js79009234YU7" puts the product 128 and later 129 into the shopping basked.

http://event.yoochoose.net/api/00000/basket/Js79009234YU7/1/128
http://event.yoochoose.net/api/00000/basket/Js79009234YU7/1/129

To buy selected products the user "Js79009234YU7" logs in and gets an internal identifier "maxmustermann" (e.g. registration ID) from the site. Do not forget that it must be URL encoded.

http://event.yoochoose.net/api/00000/login/Js79009234YU7/maxmustermann

He buys both products from the shopping basket: the product 128 (one piece and the price of EUR 19.99) and the product 129 (2 pieces with the price of EUR 4.44 each)

http://event.yoochoose.net/api/00000/buy/maxmustermann/1/128?quantity=1&fullprice=19.99EUR
http://event.yoochoose.net/api/00000/buy/maxmustermann/1/129?quantity=2&fullprice=4.44EUR

User "maxmustermann" likes the product 133 and wants to rate it e.g. with 5 stars

http://event.yoochoose.net/api/00000/rate/Js79009234YU7/1/133?rating=5

Request Format and Parameter

Events are forwarded to the Recommendation System engine using simple "http" or "https" requests; so called RESTful-Requests [1]. Both GET and POST methods are allowed for the event tracking. Make sure that all embedded and query string parameters are URL encoded (see RFC 3986). See RFC 3986 for more information. Due to a restriction of the used HTTP server, it is not allowed to use a backslash [encoded as %5C] in the embedded parameters (the context path of the URL).

Some of the tracking parameter will be described below. Please read the event description first for understanding the use case.

userid

All the tracking events require a user ID. This ID must define a user in the web presence non ambiguously. Every alphanumeric combination can be used as the user ID, therefore a registered login id can be used as a user ID. Usually a user starts a session without logging in. Until the user signs in/up, the session ID (no problem, if this ID is very long) or some other generated temporary ID (first party cookie) can be chosen as a user ID. After a user signs in, the temporary ID should be linked to the permanent user ID using the login event.

Internally the recommendation engine creates a hash of every user ID. The original ID is not saved by the recommendation engine. It is still possible, that the original ID appears in the log files for debug purposes. But log files are are purged regularly.)  The user ID is case sensitive.

itemtypeid

The recommendation engine can handle different domains of items. It is especially useful, if different types of the context are provided by the web presence, e.g. articles and images. Multiple item types are available only for the advanced edition of the Recommendation System. For a basic edition just use "1".

More information about the item type is here:
User Guide. Chapter 12. Item Types

itemid

An item ID must non-ambiguously identify the product in the web presence. Every item type uses its own ID scope. It is possible to have different items of different item types but with the same item ID. Do not use thousand separators or other symbols in the item ID, by default only integers are allowed.

categorypath

It defines the category where the item is located in. The category path is used for item filtering. For example "recommend only items from the same category".

During recommendation requests the category path must be provided in any case! The category path is a forward slash separated list of categories from the root like "%2FFoto%2FCameras%2FCompact%2FCanon". First slash (if set) is ignored. As all other parameters the category path must be URL encoded and due to some limitation of the HTTP server must not contain backslashes.

The "categorypath" parameter plays an important role as it offers the possibility to provide category based recommendations without an explicit export of the structure of a customer's website. It is used for on-the-fly updating of items meta information stored in the item stope. If the item is moved to another category, it will be handled as being in both categories until the old category ages out or is forcibly deleted.

Category Paths

If you are using a advanced solution and import your own item metadata over the recommendation engine import interface, you do not necessarily need to provide the category path with the "click" event. As this is activated by default, please contact our support (support@yoochoose.com) to set the desired configuration.

Replaying/Importing historical user data

We support replaying historical user events by adding a special parameter overridetimestamp on the buy event to simulate the event date. This is then used as the datetime of an event ant not the current timestamp.

To give an example: It is often requested to import the buy history of users to avoid the cold start problem and being able to start with good recommendations. You do not have to wait to build user profiles by collecting events from the day you implemented the tracking. Usually a curl input file is generated which creates buy requests that are sent to our tracking endpoint event.yoochoose.net.

User identifiers

Assure that the user identifiers being used in an import file are the same which are used in the live tracking later on, e.g. the user login identifiers!

There must be some validation to avoid importing buy events by any user. Therefors a signature param needs to be added which is calculated like following:

String signature=md5(encode("<itemtype>&<itemid>&<licensekey>&fullprice=<fullprice>&overridetimestamp=<timestamp>&quantity=<quantity>"));

Params without a value

The order of the params to be signed is crucial. They must be sent lexicogrphically based on the parameter names to create the signature. Encodings must always contain capital letters (e.g. %3A and not %3a)

If a param value is "" or "nul"l or "undefined" it needs to be excluded from the signing and must not be sent in the querystring!

valid: http://event.yoochoose.net/api/00000/buy/maxmustermann/1/11?fullprice=19.99EUR&overridetimestamp=2012-01-01T11%3A00%3A00&quantity=1&signature=d0026f017ae823f19530d93318c5a2f6

invalid: http://event.yoochoose.net/api/00000/buy/maxmustermann/1/11?fullprice=19.99EUR&overridetimestamp=2012-01-01T11%3A00%3A00&quantity=&signature=d0026f017ae823f19530d93318c5a2f6

Examples:

Given a licensekey of "8695-1828-92810-5535-4239" and a purchase of certain products, we have the following signature values:

http://event.yoochoose.net/api/00000/buy/maxmustermann/1/11?fullprice=19.99EUR&overridetimestamp=2012-01-01T11%3A00%3A00&quantity=1&signature=d0026f017ae823f19530d93318c5a2f6
http://event.yoochoose.net/api/00000/buy/maxmustermann/1/94?fullprice=23.99EUR&overridetimestamp=2012-01-02T11%3A00%3A00&quantity=5&signature=53ae5744879d5a3ae833f3ef34109b44
http://event.yoochoose.net/api/00000/buy/maxmustermann/1/78?fullprice=7.59EUR&overridetimestamp=2012-01-03T11%3A00%3A00&quantity=1&signature=d73e76754fb333e9733936bbd11bb5cd
...

 

If you send this big amount of requests within a loop, please inform us upfront to figure out if we need to scale the infrastructure on-demand.

Response Handling

The following HTTP response codes are used by the event tracker.  

HTTP Status Code

Description

200 OK
204 No Content

Request was successfully processed.

400 Bad Request
414 Request-URI Too Long

The request is wrongly formatted. See response body for more information.

401 Unauthorized

Illegal authentication credentials (not implemented yet).

403 Forbidden

Access denied (not implemented yet).

404 Not Found

The customer ID (a.k.a. mandator ID) was not found. The event code was not found.

500 Internal Server Error

Some unspecified error. Please inform support@yoochoose.com, if it you get this error repeatedly.

In case of errors the body of the response contains human readable error messages. Human readable error messages can be changed and should not be used for automated processing.

Integration Best Practices

There are several ways to integrate the event reporting into the webpage.

Tiny Image (server generated)

The simplest way is already described in the beginning of this chapter. One must generate a code of a tiny image and put it on the webpage everywhere (pixel tracking), where the event must be sent. For example: 

 

<img href="http://event.yoochoose.net/ebl/00000/click/max.musterman/1/100?categorypath=/a/ab/abc" width="1" height="1">

or by javascript Image tracking

<script type="text/javascript">
var img = new Image(1,1);
img.src = "http://event.yoochoose.net/ebl/00000/click/max.musterman/1/100?categorypath=/a/ab/abc";
</script>

The drawback of this option is that such a call can be blocked by ad blockers or no-track plugin on the client side.

Server side tracking

Another option is to call the tracker from server. The most important drawback is the request time of the increases by the time of the recommendation request. If the network is overloaded or the Recommendation Engine is not available the amount of requests could grow and lead to a stalled  and finally crashing httpd service. There are several techniques, which help to avoid it.

Tracking in the bottom

One can place the code at very end of the generating script and flush the output buffer to the client just before sending events. The connection to the browser will be usually still open during processing, but it will be transparent for the end customer.

Tracking asynchronously

If the website is implemented in some language which supports multithreading, non-blocking I/O or messaging infrastructure, it is possible to start the recommendation request just after the browser request is received or/and do not wait until this process finishes.

Client side tracking

JSONP

Another solution is to provide the proxy on the server side, which will forward script requests to the Recommender Engine. In this case the requests will be triggered from the client, when the page is already loaded and rendered. It is not possible to request the recommendation controller server directly from the javascript (over AJAX library or directly over XMLHttpRequest) because of the cross-domain restriction in most browsers. One of the possible technique to work around this limitation is JSONP [3].

Using a server proxy

Another option is to tunnel the JavaScript request throw the proxy on the same server. The server must just forward them to the Recommender Engine. It can be simple implemented apache proxy module, some independent daemon like “netcat” or a PHP script.

Comparison

An overview of pros and cons for every technique.

Problem
Image

Server Side

Bottom Reporting

Async. Reporting

JSON
XMLHttpRequest + Proxy
It is not blocked by ads blockers or no-track plug-ins. (tick)(tick)(tick) (tick)
It works, if JavaScript is disabled.(tick)(tick)(tick)(tick)  
It is compatible with frontend caching on the server.    (tick)(tick)
It does not delay the page rendering.(tick) (tick)(tick)(tick)(tick)
It supports authentication for event tracking (not implemented yet) (tick)(tick)(tick) depends

What we recommend

 

We suggest using Image/Pixel tracking for non-complex events and where every page is generated on the server side without caching logic. Some hints how to use preload image URLs with (var img = new Image(); img.src="uri") or without (<img src="uri"... /> javascript elements are given under http://www.mediacollege.com/internet/javascript/image/preload.html

If you are planning to implement caching mechanisms and more complex events like consume (depending on viewing time of the page), basket (which is usually an in-page event) or custom in-page events that take place on the client side we definitely recommend a javascript implementation. We provide an example script and some instructions under Tracking with yct.js

 

External Links

[1] Representational state transfer
http://en.wikipedia.org/wiki/Representational_state_transfer
[2]

Uniform Resource Identifier (URI): Generic Syntax
http://tools.ietf.org/html/rfc3986 

[3]

JSONP - Wikipedia
http://en.wikipedia.org/wiki/JSONP 

 



 

  • No labels