Friday, 26 December 2014

Websphere commerce DynaCache Part2

Please read through the post Websphere Commerce DynaCache Part1 before  reading the below section. 

 

Cache Invalidation :

Essentially it is the mechanism of removing the stale items from cache. Here is a list of the different ways you can invalidate cache:

1. Manual Invalidation
2. Time to Live (TTL)/Time Out based Invalidation
3. Dependency IDs based invalidation can be done in the below three methods.
a)Using command invalidation
b)Using CACHEIVL Table Invalidation
c) Programmatic Invalidation 


Invalidation rules
Invalidation rules inform DynaCache when and how to remove objects from the cache. They are defined in exactly the same manner as dependency IDs and use the <dependency-id> definitions to work out which objects to evict.

1. Manual Invalidation

In manual invalidation user has to go to Cache monitor (http://host_name:port/cachemonitor)and clear the cache entries.

2. Time to Live (TTL)/Time Out based Invalidation

The easiest way to invalidate cache entries are with time-based elements. This method is useful for user-specific objects when you cannot invalidate the cache entries by any other mechanism. You can accomplish time-based invalidation by specifying the <timeout>value</timeout> sub-element within a cache-entry in the cachespec.xml file. Value is the amount of time, in seconds; the cache entry is kept in the cache. The default value for this element is 0 (zero), which indicates this entry never expires.
You can use the <inactivity>value</inactivity> sub-element to specify a time-to-live (TTL) value for the cache entry based on the last time the cache entry was accessed. Value is the amount of time, in seconds, to keep the cache entry in the cache after the last cache hit. <priority> element is used to specify the priority of a cache entry in a cache. The priority weighting is used by the least recently used (LRU) algorithm of the cache to decide which entries to remove from the cache if the cache runs out of storage space.

<cache>
<cache-entry>
<class>servlet</class>
<name>/TestPage</name>
<cache-id>
<component id="action" type="parameter">
<required>false</required>
</component>
<timeout>6000</timeout>

<Inactivity>3000</inactivity>
<priority>1</priority>
</cache-id>
</cache>

3. Dependency IDs based invalidation can be done in the below three methods.

a) Using command invalidation

Let us take the example of Product Display JSP page to understand how the command invalidation works.Cache-spec.xml definition for product display is shown as below.


<cache-entry>
<class>servlet</class>
<name>ShoppingArea/CatalogSection/CatalogEntrySubsection/CachedProductDisplay.jsp</name>
<property name="save-attributes">false</property>

<!-- Cached ProductDisplay.jsp -->    
<cache-id>
<component  id="storeId" type="parameter">
<required>true</required>
</component>
<component  id="catalogId" type="parameter">
<required>true</required>
</component>
<component  id="productId" type="parameter">
<required>true</required>
</component>
</cache-id>
              
<!-- Used for invalidating the product display cache entry that belongs to a specific catalog in the store    -->     
 
<dependency-id>storeId:catalogId
<component id="storeId" type="parameter">
<required>true</required>
</component>
<component id="catalogId" type="parameter">
<required>true</required>
</component>
</dependency-id>

</cache-entry>

<cache-entry>
<class>command</class>
<sharing-policy>not-shared</sharing-policy>
<name>com.ibm.commerce.catalogmanagement.commands.AddCatalogDescCmdImpl</name>
<name>com.ibm.commerce.catalogmanagement.commands.UpdateCatalogDescCmdImpl</name>
<!-- Invalidate all the product page cache entries that might be affected when the catalog description is changed -->
<invalidation>storeId:catalogId
<component  id="getStoreId" type="method">
<required>true</required>
</component>
<component  id="getCatalogId" type="method">
<required>true</required>
</component>
</invalidation>
</cache-entry>   

This cache policy is to invalidate any cache entries that are identified by the same storeId and catalogId. DynaCache intercepts calls to the AddCatalogDescCmdImpl and the UpdateCatalogDescCmdImpl commands (which add or update a catalog description entry) and generates the invalidation ID based on the value returned by the getStoreId() and getCatalogId() methods. Upon execution of the commands, the DynaCache compares the invalidation IDs with each of the dependency IDs. Any cache entry for which any of its dependency IDs matches with any of the invalidation IDs is removed.
Here a dependency Id(storeId:catalogId) is created for the  ProductDisplay as specified in the cache-spec.xml file. So take the case when storeId is 10051 and catalogId is 10001.Dependency Id formed will be 10051:10001.Now we have to see how this entry will be  invalidated using command cache. The catalog description of the product will be changed on executing commands AddCatalogDescCmdImpl and  UpdateCatalogDescCmdImpl.During this process it will try creating invalidation Id based on the value returned by the getStoreId() and getCatalogId() methods. Upon execution of the commands, the DynaCache compares the invalidation IDs with each of the dependency IDs. So in this case invalidation Id formed (10051:10001) will match the dependency ID formed before.For any cache entry for which any of its dependency IDs matches with any of the invalidation IDs is removed from the cache.

Delay-invalidations property (<property name="delay-invalidations">true</property>)

By default, the invalidations occur prior to the perform Execute() command actually being called; however, you can change it to occur after perform Execute() returns if necessary using the delay-invalidations property which we discuss later.

In command-based invalidation, any data update command will call invalidations before the performExecute() method is called. Since DynaCache is multithreaded, it is possible that another data fetch command could potentially read data immediately after an update command finishes its invalidation calls, but before the update command commits its changes to the back-end database.

This is a problem. The fetch command would see the data that it has just read (which is actually stale) is not in the cache, and so it places it back in there. The  scenario results in cache entries with stale data that do not get invalidated as  intended. To solve this problem, the delay-invalidations property is set in the cache  policy. The delay-invalidations is used to delay command invalidations until after the performExecute() method. You use delay-invalidations at the cache-entry level in a policy for a command resource  to delay all invalidations done by it.

<cache-entry>
<class>command</class>
<sharing-policy>not-shared</sharing-policy>
<name>com.ibm.commerce.catalogmanagement.commands.AddCatalogDescCmdImpl</name>
<name>com.ibm.commerce.catalogmanagement.commands.UpdateCatalogDescCmdImpl</name>
<!-- Invalidate all the product page cache entries that might be affected when the catalog description is
changed -->
<invalidation>storeId:catalogId
<component  id="getStoreId" type="method">
<required>true</required>
</component>
<component  id="getCatalogId" type="method">
<required>true</required>
</component>
</invalidation>
<property name=" delay-invalidations">true</property>
</cache-entry>    

b) Using CACHEIVL Table Invalidation

Another way to configure automatic cache invalidation is using the WebSphere Commerce CACHEIVL table in combination with database triggers. Websphere Commerce uses a scheduled command (DynaCacheInvalidation)to check a table called CACHEIVL. You would setup triggers which would insert an invalidation id into CACHEIVL when the target table is changed.

Create database triggers to populate the CACHEIVL table. Here we are populating the CACHEIVL table with the help of a trigger when there is an update to CATLOG table as shown below.

CREATE TRIGGER DYNA_CACHE_TRIGGER
AFTER UPDATE ON CATALOG
REFERENCING OLD AS N FOR EACH ROW MODE DB2SQL
INSERT INTO cacheivl (template, dataid, inserttime)
(SELECT NULLIF('A', 'A'), 'storeId:' ||
RTRIM(CHAR(storecat.storeent_id)), CURRENT TIMESTAMP
FROM storecat
WHERE storecat.catalog_id = N.catalog_id);

Let us understand this trigger in detail. In this above example an entry is created in CACHEIVL table as follows.

TEMPLATE column

NULLIF('A', 'A')

The NULLIF function takes two arguments. If the two arguments are equal, then NULL is returned. Otherwise, the first argument is returned.So in this case NULL is returned.

DATA_ID column

RTRIM function removes all space characters from the right-hand side of a string.Hence the value in the DATA_ID column will be storeId:10001 where 10001 is the StoreID.

Since If the DATA_ID column is set and the template name is not set or it is null , then the DynaCacheInvalidation command calls the DynaCache invalidation API (invalidateById) and uses the DATA_ID as the dependency ID of the cache  entries to invalidate.

More details on TEMPLATE and DATA_ID columns of CACHEIVL table

The WebSphere Commerce scheduler runs the DynaCacheInvalidation command at a set interval. This command processes the entries in the CACHEIVL table as follows:The clearall string value in the TEMPLATE or DATA_ID columns of the CACHEIVL table is used by DynaCacheInvalidation to clear the cache using the DynaCache invalidation API (clear).

If the TEMPLATE column is set, then the DynaCacheInvalidation command calls the DynaCache invalidation API (invalidateByTemplate) and uses the  name as the template ID. If the clearall string value (which is case insensitive) is found in the TEMPLATE column, then the DATA_ID column is ignored and the DynaCacheInvalidation command clears the cache. If the TEMPLATE column is not empty, the command invalidates using the template ID, ignoring the DATA_ID column.

If the DATA_ID column is set and the template name is not set, then the DynaCacheInvalidation command calls the DynaCache invalidation API (invalidateById) and uses the DATA_ID as the dependency ID of the cache entries to invalidate. If the TEMPLATE column is empty and the clearall string value is found in the DATA_ID column, then the command clears the cache.When the DynaCache invalidation API is called, it invalidates the cache entries.

Scheduler information of  'DynaCacheInvalidation' job can be found as below.

Select * from schconfig where SCCPATHINFO='DynaCacheInvalidation';

c) Programmatic Invalidation 

Cache objects are also invalidated programmatically. This is done using the DynamicCacheAccessor object in the DynaCache API.

import com.ibm.websphere.cache.DynamicCacheAccessor;
import com.ibm.websphere.cache.DistributedMap;

if (DynamicCacheAccessor.isCacheEnabled()) {
DistributedMap map = DynamicCacheAccessor.getDistributedMap();
map.invalidate(cacheKey);
}

The static method isCacheEnabled() of the DynamicCacheAccessor object is used to find out if caching has been enabled and getDistributedMap() retrieves the cache. The invalidate method using a cacheKey invalidates cache entries. The cacheKey is either a dependency ID or a cache entry ID. If the cacheKey is a dependency ID, then all cache entries having a matching dependency ID are invalidated. If the cacheKey is a cache ID, then only that cache entry is invalidated

There are different ways by which we can do the invalidation programmatically as shown below.

1. First Approach 
DistributedMap distributedMap = DynamicCacheAccessor.getDistributedMap();
distributedMap.invalidate(dependencyId, false);


2. Second approach
com.ibm.websphere.cache.Cache dmap = com.ibm.websphere.cache.DynamicCacheAccessor.getCache();
if(dmap != null){
dmap.invalidateById("<<dependency-id>>", true);
}


3. Third Approach
com.ibm.websphere.cache.DistributedMap cache = com.ibm.websphere.cache.DynamicCacheAccessor.getDistributedMap();
if(cache != null){
cache.clear();
}