3

Building High Speed Cache

Posted by
at December 19, 2010

Recently while developing a caching tool capable of providing access to millions of records in sub-seconds, we analysed some of the existing best practices.

(1) One choice is MySql native memory cache for avoiding GC-pauses.

MySQL InnoDB storage engine’s dictionary cache provides blazing fast data acceess. On demand, data blocks are read from disk and stored in memory using LRU chains or other advanced mechanisms(like MySQL’s Midpoint Insertion Strategy) .

When should we use MySQL query cache?

Here is the summary from – http://dev.mysql.com/tech-resources/articles/mysql-query-cache.html

  • Identical queries are issued by the same or multiple clients on a repetitive basis.
  • The underlying data being accessed is static or semi-static in nature.
  • Queries have the potential to be resource-intensive and/or build brief, but complexly computed result sets.

Data warehouse/business intelligence situations, web-based applications, and traditional OLTP systems all qualify on the surface. The query cache looks for identical queries (spacing, upper/lower case, all come into play), ergo the first point above.

For internals of InnoDB Cache, refer to the MySql  tutorial.

In general, MySQl memory cache provides best performance if the majority of your database queries are simply SELECT statements. Here the speed gain is definitely due to negligible GC pauses and direct disk access. This type of caching strategy is popular for eCommerce and bidding platform where data query is more frequent than data updates.

(2) An alternative solution is ‘Memcached’ which lives in a different process and manages its own GC (optimized for caching). We know that in order to lookup the value for a key,  Memcached cache clents issue multiple parallel requests to memcached servers where keys are sharded across those servers.

The value-items resolved by Memcached are usable objects while those returned from the MySQL cache must then be converted into objects.

As queries get more complex, Memcached provides more efficient performance.

If an application (using mysql cache) is load-balanced over multiple servers, then each web process will maintain a separate duplicate cache unless otherwise customized and optimized specifically.  But the advantage of using Memcached is that it creates a single global cache available to all web processes . Individual web processes and threads do not maintain their own caches in this case, but rather use the single global cache. When one web process puts data in the cache, it becomes available for all others to access. It leverages fast Network connections.

The detail architecture of Memcahed is out of the scope of this article.

For a frequently changing data set , Memcahed is surely better than DB Cache as ‘bulk of the data are read from memory rather than disk’ . But point to note – it still suffers from GC-pauses however minimal it could be !

(3) Now lets see how ‘Ehcache’ (leveraging Terracotta BigMemory) has really got the basics right !

Just by simply adopting a very well known Disk I/O pattern of accessing native memory directly from java, it has simply eliminated the ‘GC’ factor and proved to be the fastest caching option !

Now the choice is ours – (a) whether to adopt a caching strategy that requires “build a bigger computer with huge RAMs” (b) or use the faster cache in a machine with smaller RAM.

The bottom line is – the cache must be outside GC’s control. There is no GC mechanism available as of now (unless we leverage JDK 7 ‘Garbage First’ ) - which provides a relief from ‘Stop-of the World’ stuation.

The trick is to implement a smart serialization technique by storing key, value pairs into Direct Byte Buffer.

Here are some insights - http://www.ibm.com/developerworks/java/library/j-nativememory-linux/

Direct ByteBuffers (allocated using the java.nio.ByteBuffer.allocateDirect() method) that are backed by native memory rather than Java heap. Direct ByteBuffers can be passed directly to native OS library functions for performing I/O — making them significantly faster in some scenarios because they can avoid copying data between Java heap and native heap.

It’s easy to become confused about where direct ByteBuffer data is being stored. The application still uses an object on the Java heap to orchestrate I/O operations, but the buffer that holds the data is held in native memory — the Java heap object only contains a reference to the native heap buffer. A non-direct ByteBuffer holds its data in a byte[] array on the Java heap.

A small prototype using ButeBuffer really yielded high performance.

Here is a sample code for creating direct-buffer-based cache (of course its just for learning, no where close to actual ehcahe implementation). http://tusharkhairnar.blogspot.com/2010/09/bigmemory-memory-with-no-garbage.html

This is how Terracotta Bigmemory got the basics of caching right by implementing ByteBuffer and bypassing GC pauses ! http://blog.terracottatech.com/2010/09/bigmemory_explained_a_bit.html

ConcurrentMarkSweep collectors can run for hours w/ no full pauses and then suddenly stop the world for 30 minutes trying to catch up to the mess they have made.

…. if (k1, v1) is in cache in Ehcache BigMemory, it has no references to (k2, v2). They cannot refer to each other. Which means I don’t have to walk the cache looking for cross-references in the data like a GC does. No mark-and-sweep. I merely wait for the dev to call cache.remove( element ) or for the evictor to evict stale data. The evictor and the remove() API are signaling my MemoryManager explicitly just like new() and delete() or malloc() and free(). There is no notion of garbage or collection. Only memory re-use and fragmentation over time.”

If we want to enjoy the fruits of fastest cache we have to perform some simple steps :

http://ehcache.org/documentation/offheap_store.html

(i) Update the Java classpath to include the new ehcache jar

(ii) Modify the App Jvm settings to allocate enough direct byte buffer

-XX:MaxDirectMemorySize affect the memory available with this call java.nio.ByteBuffer.allocateDirect(int).

(iii) Update the ehcache.xml to set the size of the

BigMemory off-heap store.

<ehcache>

<cache name=”mycache” maxElementsInMemory=”10000″

overflowToOffHeap=”true” maxMemoryOffHeap=”4G”/>

</ehcache>

Here in this link - http://code.google.com/p/lucene-bytebuffer/ , we can see how Lucene Buffers bypasses memory intensive slow java garbage collector.  The objects whose life-cycle is known beforehand can be effectively cleaned up through simple manual process instead of relying on GC.

The basic advantages of a direct-buffer-based cache (like BigMemory) is :

1. Bigmemory takes away all cache data from gc-managed memory to its own bytebuffer-managed world !

2. No need to tune GC

3. Low hardware cost and operational simplicity. User can have one 16 GB RAM and and make use of all that memory with only 1 JVM. This saves the user from the huge effort of maintaining multiple JVMs to create a distributed cache. User can still go fot it and utilize every byte of the RAM without waiting for GC pause.

On a different note, though any one can implement a fast cache (as discussed above) using nio package in jdk 1.6; but if the app involes heavy file handling (index files or web resources or jars) then for faster experience we should use jdk 1.7 nio2.

Here are reasons : (http://download.oracle.com/javase/tutorial/essential/io/legacy.html#mapping )

Prior to the JDK7 release, the java.io.File class was the mechanism used for file I/O, but it had several drawbacks.

  • Many methods didn’t throw exceptions when they failed, so it was impossible to obtain a useful error message. For example, if a file deletion failed, the program would receive a “delete fail” but wouldn’t know if it was because the file didn’t exist, the user didn’t have permissions, or there was some other problem.
  • The rename method didn’t work consistently across platforms.
  • There was no real support for symbolic links.
  • More support for metadata was desired, such as file permissions, file owner, and other security attributes.
  • Accessing file metadata was inefficient.
  • Many of the File methods didn’t scale. Requesting a large directory listing over a server could result in a hang. Large directories could also cause memory resource problems, resulting in a denial of service.
  • It was not possible to write reliable code that could recursively walk a file tree and respond appropriately if there were circular symbolic links.

Jdk 1.7 has great many changes to improve developer’s productvity and app’s performance – (http://tech.puredanger.com/java7/ )
The last but not the least, since we are discussing handling of direct byte buffers, its worth keeping in mind the recommendations posted at – http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly

(a) Minimize I/O operations by reading an array at a time, not a byte at a time. An 8Kbyte array is a good size.
(b) Minimize method calls by getting data an array at a time, not a byte at a time. Use array indexing to get at bytes in the array.
(c) Minimize thread synchronization locks if you don’t need thread safety. Either make fewer method calls to a thread-safe class, or use a non-thread-safe class like FileChanneland MappedByteBuffer.
(d) Minimize data copying between the JVM/OS, internal buffers, and application arrays. Use FileChannel with memory mapping, or a direct or wrapped array ByteBuffer.

References :

http://blog.terracottatech.com/2010/09/bigmemory_explained_a_bit.html

http://ehcache.org/documentation/offheap_store.html

http://www.ibm.com/developerworks/java/library/j-nativememory-linux/

http://developers.sun.com/learning/javaoneonline/2008/pdf/TS-5419.pdf

http://tusharkhairnar.blogspot.com/2010/09/bigmemory-memory-with-no-garbage.html

http://download.oracle.com/javase/tutorial/essential/io/legacy.html#mapping

http://java.sun.com/developer/technicalArticles/javase/nio/

http://tech.puredanger.com/java7/

Uncategorized
0

Benefits of Moving to Cloud

Posted by
at November 19, 2010

A summary of Benefits of Cloud is listed below.

Uncategorized
0

Delivering again and again!

Posted by
at November 19, 2010

Delivering value again and again

Business needs to optimize their spending and still get greater value over time with the money they spend on services. ISV’s venturing into offshore development has to evaluate their vendors with parameters that measure the value of the vendor more quantitatively. They could also use empirical methods to determine the value vendors bring onto the table.  Bigger ISV’s tend to go for captive centers long term to protect their IP and reduce dependency and lock into the vendor. They also recognize the additional support staff, infrastructure required to invest. There are no tools to measure these choices to make a solid decision.

History has proven the value of services oriented organization and businesses are reorganizing to adapt to services oriented approach. When posed with a challenge of choosing an external organization that delivers business value compared to an internal org, businesses have to make a tough decision.  Businesses are bowing to pressure and compromising on time to market, value to business and long terms benefits.

Software and services vendors are evaluated for certain criteria. Almost all the time, cost is the most important factor, which drives the deals in the software industry compared to any other driver.

An ISV vendor chooses an outsourcing partner by evaluating various factors.

Most of the ISV vendors prefer captive centers for their long term strategy. The strategy could end up leaving the vendor with resources who are not very effective. Over time the weight of the supporting organizations, logistics go up and reduce the effectiveness and productivity of people.

The ability to hire and retain top talent is number one challenge in these organizations. However, the services approach allows the captive centers to choose and demand the best from the vendors. This strategy may cost little more in the beginning and lot less in terms of total cost of ownership, if we consider intangible benefits like time to market, innovation, disposability etc.

The cost of a change goes exponentially up, when the product is in production or lives. This is very much a problem in captive centers, where they can simply assign people to fix issues. However, over time, the code quality suffers and the product becomes difficult to maintain and every change takes 3 to 6 months to get into production. The value that Product Development vendors bring in here are enormous, because of their ability to innovate and provide solutions. They do a better job of refactoring code, because better code and better quality for them means better business.

We at Imaginea deliver 150% productivity, because we have been able to operate at startup mode for every single customer we have worked with. We are able to deliver because; we have mastered the art of making software. We are able to deliver because we have been doing this for 12 years and that is our bread winner. We are able to replicate our success to small big and enterprise customers regardless of the size, the same value again and again.

At Imaginea, we provide you with an extra an edge over other vendors to beat the competition.

Cloud Computing, Offshoring Product Development, QA, SaaS Engineering, Services, Software Product Testing, Uncategorized
12

Automating SOAP API tests with SOAPUI

Posted by
at September 16, 2010

SOAP UI is a powerful and easy to use open source tool for automating different types of web services like SOAP, REST etc. It provides a UI for creating test cases,assertions making it easier to develop test cases rapidly.

SOAP UI caters to the needs of functional, regression, load testing and provides  features like

This blog covers basic steps and information on how to start off with SOAP UI, create basic test cases and assertions and execute the test cases.

Install SOAP UI

Install the latest free version of SOAP UI from http://www.soapui.org/

Create a project

  1. Open SOAP UI.
  2. Click on file->new SOAP UI project
  3. Provide the URL to the WSDL in the project creation window and click OK. A service binding for the WSDL will be generated with all operations.

When the WSDL for a services changes, it can be updated without affecting the test cases by right clicking on the service binding and choosing update definition.

Generating test suite/test case

  1. Right click on the service binding and select “Generate TestSuite”. Select the operations for which sample test case template has to be generated. A test suite will be generated for the selected operations.
  2. Right click on the test suite to generate a new test case

Adding steps in a test case

When a new test case is added, right click on the test case and select add step -> test request.Select the operation for which the request needs to be sent.

Once test case is added, different steps to the test case can be added.These steps form the blocks of the test case which are executed in order.

How to pass parameters to tags

Values for the tags in the SOAP request can be hard coded or parametrized. There  are two ways of doing passing parameters/values

Creating properties to transfer values: Properties can be used to store data. These can act as variables which can be passed to the SOAP requests.

Properties at a project : Properties can be defined at project and these are available for all the test suites and all test cases. Double click on project and click on properties tab at the bottom to create properties

Properties at the test suite level : Properties which are defined at the test suite level are available for all the test cases and test steps inside the suite. Double click on test suite and click on properties tab at the bottom to create properties

Properties at the test case : Properties at the test case are available for only that test case and all the test steps inside the test case. Right click on test case and select add step -> properties to create properties for a test case.

Declaring properties using properties test step

Declaring properties

Transferring properties using properties transfer test step

Transferring properties

Data driven by external files: Is available only in the commercial version

Creating assertions

Assertions are at the heart of any test case where the actual testing happens

SOAP UI provides different types of assertions. Double click on a test request inside a test case . You will find an assertions tab at the bottom and different kinds of assertions can be added.

Assertions

Running the test cases

Running from UI :

Double click on a project or test suite or test case and click on run button to execute tests at that level. Passed test steps in each of the test suite are shown in green and failed ones in red.

Running test cases from UI

Running from command prompt:

SOAP UI test cases can also be run from command prompt by using the following command

com.eviware.soapui.tools.SoapUITestCaseRunner (options)

Few options available

-s   Test suite to run

-c  Test case to run

-r  Turn on printing of summary report

Another easier way is to run the tests is using the testrunner batch file which comes along with the SOAP UI installation at /bin

testrunner.bat -sTemperatureServiceBindingTestSuite -j -f E:\SOAPUITestReport E:\SOAP-API-Automation-soapui-project.xml

-s is for the test suite

-j indicates that junit style HTML report will be generated

-f is for speficying the folder for report generation

Mentioning the location of project XML is mandatory

Running from ANT:

An ANT tagert can be created which executes this command line task to integrate with the builds.


<target name=”runSOAPUISuite”>
<exec executable=”C:\Program Files\eviware\soapUI-3.5\bin\testrunner.bat”>
<arg line=’-sTemperatureServiceBindingTestSuite -j -f E:\SOAPUITestReport E:\SOAP-API-Automation-soapui-project.xml’/>
</exec>
</target>

Importing the test cases

The test suites and projects can be exported and imported to different machines and thus can be shared and executed by other team members very easily as it provides the same view on import.

QA, Software Product Testing
0

true hotswap

Posted by
at September 16, 2010

Like many other java users, for years, I have been yearning for true hotswap. I want to add or remove methods, fields, interfaces and everything else like that at runtime and my vm in debug mode should be smart enough to put it in without complaining about non-binary compatible changes. If you have already heard about “Dynamic Code Evolution VM”, you can skip this whole article, as it adds nothing new, but if you haven’t ….

Now thanks to the effort by some really smart guys at Institute for System Software at the Johannes Kepler University Linz, this is possible. We can find the binaries here. The jar file acts like an installer which searches for the vm (or you can point one to it) and then soups up the jre to provide the better hotswap. If we want to revert to the original vm, we can uninstall this patch via the same installer jar. This installer can enhance 1.6 jres too.

Since the actual implementation does need to make some changes to the core pieces, internally it replaces the libjvm.xx with its own version. This change can be undone if we uninstall the patch. We can find the source code here. This source is a bit back dated, but gives us an idea of what the actual changes are.

The way it works is when a new version of the class is fed to the vm, it loads the new version and switches all usages to it. The class structure has been modified to maintain multiple versions of itself.It also takes care of updating the existing objects with the new fields and methods. To do so, it does make some real changes at the method and interface table level as well as the way classes are stored. (oversimplification alert).

In case of enterprise applications, it is possible that we may need extra operations to be done on these, if that is the case we can use something like classfiletransformers to know which classes have been changed and then make the proper changes for it. For example if we change a spring bean and introduce a new dependency, we can detect that the transformed class is a spring artifact and then update this bean with the new dependency (my version schedules a task from the classfiletransformer which executes a few seconds later that looks up the same bean in spring context and wires the new dependency field).

I seldom blog about solutions in which I have no hand, but this is an exception, as I think there should be a lot of noise about this feature and a lot of appreciation for Thomas Wuerthinger, so am doing my bit.

P.S: If you are thinking about java rebel, the approaches are very different, so go through the source to get a better understanding before drawing any conclusions or comparisons.

Java
0

Perils of URLClassloader

Posted by
at August 26, 2010

Its a very common programming technique to create any number of URLClassLoader instances repeatedly in order to load the new implementation of the classes/resources from the same location with the help of new class loader instances.

In fact, some well-known tools like plexus-compiler jar makes heavy usage of IsolatedClassLoader to implement the above concept.

Lets focus on the actual problem.

We have a large maven project which has 200 jars in maven repo.
Any code change in source file fires – MavenBuilder …..
- which invokes JavacCompiler (plexus-compiler jar)
- in turn creates IsolatedClassLoader cl = new IsolatedClassLoader() [extends URLClassloader] ..
- adds the list of 200 jars in urlClassPath  - cl.addURL(URL)
- loads javac.Main along with all jars in classpath and reflectively fires compilation.

So for N save-operations, N instances of IsolatedClassloaders are created.

In principle, once the application clears all references to a loader object, the garbage collector and finalization mechanisms will eventually ensure that all resources (such as the JarFile objects) are released and closed.

But in reality, the application goes OutOfMemory !

We used optimal concurrent gc strategy - Xgcpolicy:optavgpause.

But still the app running out of memory very quickly.

Then Heap dump analysis showed there are some 100 instances (Uuh !) of isolatedClassloaders each of  which is holding onto unclaimed ZipFileIndexEntries(.. guess what .. indexEntry for all the jars loaded on each instance of class laoder..)

Well ! looks like.. since a new URL Class Loader is created before closing the resources in previous loader, GC is confused and does not reclaim the previous class loader ! This causes problems for applications which need to be able to GC in a predictable and timely fashion. It is a particular problem on Windows, because open files cannot be deleted or replaced.

So are we missing something trivial here ? Is plexus-jar doing something wrong ?

A BIG Emphatic YES !

We should always use sun.misc.ClassLoaderUtil to release an url class loader and file resources held therein.

Lets now fix the issue.

org.codehaus.plexus.compiler.javac.JavacCompiler#compileInProcess(..) {

IsolatedClassLoader loader= new IsolatedClassLoader();

loader.addURL(jarListoURI().toURL())

c = loader.loadClass( “com.sun.tools.javac.Main” );

ok = (Integer) compile.invoke(args)

//Now that you are done with compilation, get rid of loader ..

*** loader.close () Or sun.misc.ClassLoaderUtil.releaseLoader(loader)

////// -> allows graceful release of loader and all resources

*** Thankfully JDK 7 URlClassloader has implemented close() method  to give the caller a chance to invalidate the loader, so that no new classes can be loaded from it. It also closes any JAR files that were opened by the loader. This allows the application to properly delete or replace these files and, gracefully create new loaders using new implementations.

The bottom line is wherever we create seemingly harmless URlCLassloader in a repeated manner (say to generate / compile code) holding onto a good many jars in url classpath; we should always close the loader instance.

Java
0

The most advanced software factory!

Posted by
at August 25, 2010

Software Development has come to a mature state with standard frameworks and technologies that could be adopted to solve a class of business problems and or with specified set of architectural stacks. With most recent software development processes like SCRUM, extreme programming, and the customers are closely working with the team to get the software delivered to customers more effectively. The software development and delivery has become more mature and it is akin to manufacturing industry where assembly lines rollout vehicles continuously, except in software development, the assembly line is the sprint that delivers workable, testable and usable software.

In such a scenario, what are the challenges? The line of business managers are constantly presented with challenge of choosing the right partner, right vendor to source. The process of evaluating vendors experience with appropriate domain and technical stacks is a challenge. Other challenges include choosing an architecture that suits him based on his short and long terms goals.

There are numerous options to source work with. There are biggest like Infosys, TCS who can offer their vast experience, stability and experience. There are small players who can get to the goal in shortest and most economically way. They are also niche players who can get the software out with right parameters. The driving factors for choosing a vendor political, administrative, technical, and geographical and mostly the biggest factor is the financial viability as well.

Once the vendor is selected, you will start working with the vendor the exact process, set deadlines and quality goals and continuously monitor with metrics that could validate your choice of the vendor. Unfortunately, there is no standard for metrics and most of the time, the experience of the managers with different vendors serve as the most important metric in defining the relationship.

With more than 100 years of accumulated software development experience and a decade of experience in developing marketable products, and with four successful products under belt, we have a case to prove that, we are the most advanced factory producing software in the world. We have mastered the art of producing software. Be it be the user interfaces, be it the scalable designs we come up, be it with the flexibility of the software, be it the perspective and attitude we work with, be it with the code quality, we bring in right value.

We will help you focus on your key goals to get your product right and in time to market. Instead of worrying about development, we help you to focus on the market. This results in taking the huge burden off of your shoulders, which will allow you to focus more on business rather than day to day headaches of software development. You should not be worried about technology, execution and implementation. With the strong, agile light weight process that is customized for each customer we drive in value to right from day one till the end.

If you have an idea and want to take the market, we can help you take it to the market in the shorted passible time frame. Since, we are saying, we are a factory, we have mastered the art of producing software. We can produce software at faster, better and competitive.

You name the technology that you want to work with. Our factory has the skill and expertise to weave your product. You want to work with Social media or ERP systems, health care or advanced mining applications, you name it, we can get you there in short period and in time.

Over the years we have not only mastered the art of producing software, we have built a strong team of technology evangelists who can help you build solution with most of the latest state of the art technology stacks we have available today.

Imaginea is the destiny for most of the vendors who understands the maturity of software development, in terms of technical frameworks, products, time to market etc. We are the right partner of choice with right people, right size, and right people and with the right mind set.

A successful product development hinges on the effective sharing of ideas, leveraging organizational knowledge and ideas by its talented workforce. Effective knowledge sharing within teams takes substantial time. Imaginea is built on the fundamentals of sharing, mentoring and coaching people to become technology evangelists, entrepreneurs and business leaders.

If you are not convinced, give a call to our marketing or send us an email or talk to some of our customers.

Uncategorized
1

Successful Product Management in an Agile World

Posted by
at August 20, 2010

Having a Right product strategy is a  critical factor in making sure the desired product comes out of  agile teams. As a leader in providing product development services to ISV’s at Imaginea, we have over the period of time understood what successful product managers do to roll out world class products. Some of us have proven experience in sharing these experiences to value add to our customers. We have closely watched some of the best product managers that we have worked with, and have fine tuned the art of managing and delivering products. Our own products are a reflection of some of these practices am going to speak about.

I have been trying to understand the Agile practices for quiet some time now. Having closely examined few projects that have been adopting agile practices, i made certain notes and observations. In my free time i also interviewed few successful agile practitioners, professionals and scrum masters. I have also spoken to some senior product development engineers who have shared their experiences. Here are some thoughts:

Background/ Context

Another organization that i knew practices agile methods to roll out the product it has in mind. Recently I spoke with a team of very frustrated Product development engineers there who were using Agile methods. They were not so happy for a simple reason that, they were working for the past one year trying to finish the  features they were responsible for.But the hard part was that the product manager really had no idea where they were moving or what they were really trying to accomplish.

That was when i realized that i should talk to the product manager (being a product manager my self previously it was easy for me to reach to him). Speaking to him, i realized he is also the  product owner- as Scrum calls it.He explained that he thought the whole idea of Agile methods was to be nimble, remain flexible and agile. But he was not convinced of the fact that more important was to be sure about the long-term direction of the product.I started realizing that creating an effective product strategy may have become an unintended casualty of the move to Agile methods.

I now strongly believe we need to understand what a right product strategy is, why is it very critical for a successful product launch and its complete alignment with the Agile methods.

What is a product strategy

Loosely put, Product strategy is the vision you wish to accomplish say in 2-4 years. It is for sure not a specification or a document. It is certainly a vision the leader or the product owner must be able to rally the teams with. If this objective is met correctly half the job is done.

Product Strategy communication

Modern day product managers use a lot of collaborative technologies to communicate the strategy to their teams. It could be wiki’s, e2.0 products, presentations or the same old style documents. The critical factor here is the communication has to be clear and inspiring. It could be a choice of presenting in person or through a shared desktop presentation which will not be a differentiating factor.

What must a Right Product Strategy Address

a) The difference that the product aims to make to its intended customer base.

b) Existing problems that the customer has and how they are solved using the product in making.

c) What will be those WoW factors that the users would resonate with in using the product

d) Essentially product strategy is a link between the corporate business strategy and the road map that the product wishes to traverse. Product strategy thus must enable the achievement of business strategy and road map must help you reach there step by step.

e)A good product strategy is one of the most important things a product manager should look to have.  It’s not easy but without it you have little hope of actually ending up with something worthwhile.

f)Make sure your executive teams reviews, approves and appreciates the product strategy. Do not assume that it will come by itself from top-down. It will never happen.

g) Only if  you’re in a start up with a founder serving as the product visionary there is a chance that the product strategy gets pushed from top-down. Otherwise product manager must hold the forte and communicate the same to the management.

h) It is essential to understand that the product strategy should not bind/ restrict into any particular features or sequencing. Features and their order of implementation are represented as part of the backlog.

i) However as an alert and efficient product manager to make sure you are on top of the schedule, You can, and absolutely should, adjust the roadmap constantly
based on what you learn from your users, the market, your analysis, trends, reports, and
the ever-changing technologies that the product is being built upon.

j)Majorly impacting the product manager’s decision point is the trade-off’s that will arise during the course of the implementation journey. A product manager here must wear the hat of the customer first and engineering next. That will also ensure he understands what the marketing and sales would prefer going into to the field than pure play engineering.

k)Here i believe that Agile methods, applied properly, will help you make
your product strategy a reality. Also it would ensure its  significantly swifter than the traditional methods.It gives you the flexibility to turn around in 2 weeks per sprint if some thing is really worth changing as part of the trade-offs you encountered.

Conclusion

Having a right product strategy, implementing the strategy in alignment with the business plan and fine tuning the road-map will be the key to successful product management. product backlogs have to be effectively managed to ensure the right return on investments are in place from your agile teams. This way your engineers, your management, your partners in delivery will all be equally participative in getting the right product out.

Offshoring Product Development, Uncategorized
2

viewing dynamic methods grails

Posted by
at June 28, 2010

Grails reminds me of those Anton Chekov stories I used to enjoy. A lot of fun, but tough to remember the plethora of characters introduced at different parts of the story, in the case of grails, it is methods added via meta-programming. Each of these methods add an important functionality, but it is tough to guess where the implementation source is and where this method was added.

For example we know every domain class has a list method, but where is it added to the domain class and which class actually backs the list method implementation. If we had information about where it was added in this case HibernateGrailsPlugin, a quick browse through the source would let us know that ListWithFilterMethod is our points of interest.

So I wanted something that for a domain class would give me method listings, with the one for list like

list

public static Object list()
Added by HibernatePluginSupport$_addQueryMethods_closure44

This tells me that list method is added by addQueryMethod of HibernateGrailsPlugin. I can easily find from the source then that ListWithFilterMethod provides the list implementation. The next step is to add a controller and some ui which would help me show such information about any class (Domain, Controller or any other groovy class). The class name could be passed as a request parameter, the ui could be made via jquery so that this fits in as an additional floating div over my existing ui.

Thanks to the simplicity of grails, all this controller has to do is :

  1. Load the class from the parameter name, get its meta data
  2. Iterate through the meta methods, collecting the source from the closure or cached method declaring class and pass it to the ui as json.

The code for doing this could be something like

def metaClass =GrailsUtil.getMetaClass(params.clazzName)
metaClass?.getMethods().each{
    if (it instanceof ClosureStaticMetaMethod) {
       list.add(new MethodInfo(it.name, it.closure.class.simpleName, extractMethodName(it)))
    } else if (it instanceof ClosureMetaMethod) {
       list.add(new MethodInfo(it.name, it.doCall?.cachedMethod?.declaringClass.simpleName, extractMethodName(it)))
    } else {
       list.add(new MethodInfo(it.toString(), null, null))
    }
}
list.sort()
render list as JSON
The end result looks something like

MethodInfo

I call it EGDoc, added it as a floating layer so that it does not come in the way of normal functioning of the application, of course is enabled when the environment is development.

groovy
0

Financial Technology 3.0

Posted by
at June 27, 2010

Advent of cloud computing, virtualization, mobility and social computing paradigms have enabled newer definitions for the entire financial technology space.FT 3.0 is all about reaching the unreached. It is more about being inclusive and leveraging the un-tapped lower strata of the pyramid which is more dynamic in both consumption and spending.

Existing financial solutions across the BFSI segment are gearing up to be SaaS enabled, Cloud deployable and handset capable solutions. Micro finance systems combined with m-commerce transactions will slowly push banking services to be as dynamic as it can be.

Per Day Banking – A new Disruption

What I visualize will be a day where entire unit of transactions that a consumer does inside the financial ecosystem will be per day. Rapid adoptions of pay per use services will change the revenue streams of financial institutions. M-Wallet transactions will replace the serpentine queues at ATMS (like how ATM’s took over typical visits to Bank). Virtual currencies will move from being inside a social gaming platform to real world usage and applications. Exchange rates will be more centralized, common currencies and centralized trading platforms breaking barriers of economic adoption will erupt.

Service Delivery framework for inclusive financial services

Effective delivery frameworks that encompasses continuous integration of activity streams that arrive out of different technological convergences is very vital for FT3.0 to fully provide all that it can. Compelling return on investments, reduced carbon foot print and more- per single transaction are the key aspects that will define the results that ft3.0 can deliver.

Eg In India, Banking institutions for example are seeing an increased mandate to promote inclusive development and tap into rural, semi urban and untapped market, consumer spaces. Currently available banking solutions and frameworks may not /cannot cater to such a customer base unless they leverage the adoption of new age technologies that constitute FT3.0. A framework that can solve such a problem, catering to billions of people in India, can easily replicate itself successfully for Global markets with required customization.

Shared common Framework for Financial Institutions:

A potentially fool proof framework that needs to be envisaged is a Shared common framework that can leverage the power of cloud computing.

A shared common framework helps large number of banks to take advantage of shared infrastructure and resources, rather than each making its/their own investment. This model offers economies of scale to customers enabling banks to eliminate upfront high capital costs through sharing of application services.

This results in reduction of significant capital outlays as it will lead to a totally new and radical transparent monthly pricing model. Such a solution built on a shared common framework will catalyze efficiency in operations, provide greater compliance and risk management, enhance customer satisfaction and integrate more banks and financial institutions into main stream banking and financial activity.

Conclusion :

FT3.0 is the way forward. It will certainly be beyond boundaries. It will certainly be disruptive. Imaginea is at the fore front of this innovation.  Partner with us and stay ahead.

My other Blogs relevant to the topic:

http://blog.imaginea.com/financial-industries-and-cloud-services-trends-and-innovations/

http://blog.imaginea.com/new-age-banking-is-all-about-being-social-and-cloud-enabled/

Cloud Computing, SaaS Engineering, Services
Previous Page
Next Page