Papers

Once-in-a-decade leap

jEFF wILKE, cONSUMER dIVISION ceo, aMAZON iNC.

This neural network classifier algorithm for recommendations that I developed was 2x better than the collaborative filtering algorithms which is has been the heart of Amazon’s product recommendation for decades. The work was described as a “once-in-a-decade leap” by Jeff Wilke.

An less technical overview is provided in:

The Effectiveness of Two-Layer Neural Network for Recommendations

Oleg Rybakov, Vijai Mohan, Avishkar Misra, Scott LeGrand, Rejith Joseph, Kiuk Chung, Siddharth Singh, Qian You, Eric Nalisnick, Leo Dirac, Runfei Luo

International Conference on Learning Representations, 2018. 

We present a personalized recommender system using neural network for recommending products, such as eBooks, audio-books, Mobile Apps, Video and Music. It produces recommendations based on customer’s implicit feedback history such as purchases, listens or watches. Our key contribution is to formulate recommendation problem as a model that encodes historical behavior to predict the future behavior using soft data split, combining predictor and auto-encoder models. We introduce convolutional layer for learning the importance (time decay) of the purchases depending on their purchase date and demonstrate that the shape of the time decay function can be well approximated by a parametrical function. We present offline experimental results showing that neural networks with two hidden layers can capture seasonality changes, and at the same time outperform other modeling techniques, including our recommender in production. Most importantly, we demonstrate that our model can be scaled to all digital categories, and we observe significant improvements in an online A/B test. We also discuss key enhancements to the neural network model and describe our production pipeline. Finally we open-sourced our deep learning library which supports multi-gpu model parallel training. This is an important feature in building neural network based recommenders with large dimensionality of input and output data.

[ Paper: PDF ]


Anatomical landmark detection for image segmentation are time and processing intensive. Machine learning techniques provide significant speed up with minor drop in accuracy. Combining the two can find the best of both worlds.

Automatic Lung Segmentation: A Comparison of Anatomical and Machine Learning Approaches

Avishkar Misra, Mamatha Rudrapatna, Arcot Sowmya

International Conference on Intelligent Sensors, Sensor Networks and Information Processing, 2004

The aim of this work is to develop an automatic lung segmentation system, capable of segmenting the lung into Apical, Middle and Basal regions, along the axial plane of the body. An accurate segmentation of the lung is important for diagnosis of diffuse lung diseases, as well as to characterise and track particular diseases. In this paper, we compare the two strategies we have developed. The anatomy based approach uses anatomical landmark detection to define the separation points between the regions, whilst the machine learning approach uses lung shape, size and location properties, to classify a given lung into the appropriate region.

[ Paper: PDF ]


ProcessRDR reduced the time to develop a computer vision system from 3 months down to 4.5 hours, by adapting incremental knowledge acquisition technique of Ripple Down Rules to the learning of knowledge about controlling image process algorithms.

Incremental Learning for Segmentation in Medical Images

Avishkar Misra, Arcot Sowmya, Paul Compton

IEEE International Symposium on Biomedial Imaging, 2006

Hand-coded vision systems are problematic in complex medical domains and are hard to change as new information emerges. Knowledge-Engineering and Machine Learning approaches to intelligent vision systems also face learning bottlenecks. We have developed an approach to engineering vision systems, which allowed the user to make incremental changes to refine the performance of the system and address these limitations. A medical image segmentation system was built using this approach. In only a few hours of training, the system was able to exceed the performance of a similar hand-coded system built over a period of three months.

[ ISBI 2006 Paper: PDF ] [ PKAW 2004 Workshop Paper: PDF ]


Too much choice is a bad thing for Quasi-experts. Limiting the options for Quasi-experts by using nominal features and smart binning can reduce the errors in knowledge acquisition by 3 times.

Impact of Quasi-expertise on Knowledge Acquisition in Computer Vision

Avishkar Misra, Arcot Sowmya, Paul Compton

International Conference on Image and Vision Computing, 2009

Ripple Down Rules (RDR)’s incremental knowledge acquisition provides computer vision applications with the ability to gradually adapt to the domain and circumvent some of its learning challenges. RDR use incremental exception-based theory revision and rely on the expert to provide the rule conditions. A computer vision expert whilst understanding their significance cannot always provide accurate rule conditions using numeric attributes. This work investigates the impact of the quasi-expertise of vision experts on the structure and performance of the acquired knowledge base. The findings provide insights into the design of features and strategies to facilitate the use of quasi-expertise for knowledge acquisition in computer vision.

[ Paper: PDF ]


ProcessNet allows developers to incrementally develop a very complex vision system using simple image processing techniques and incrementally acquire or discover control and domain knowledge.

Incremental System Engineering Using Process Networks

Avishkar Misra, Arcot Sowmya, Paul Compton

Pacific Knowledge Acquisition Workshop (PKAW), as part of Pacific Rim International Conference on Artificial Intelligence (PRICAI), 2010

Engineering of complex intelligent systems often requires experts to decompose the task into smaller constituent processes. This allows the domain experts to identify and solve specific sub-tasks, which collectively solve the system’s goals. The engineering of individual processes and their relationships represent a knowledge acquisition challenge, which is complicated by incremental ad-hoc revisions that are inevitable in light of evolving data and expertise. Incremental revisions introduce a risk of degrading the system and limit experts’ ability to build complex intelligent systems. We present an incremental engineering method called ProcessNet that structures incremental ad-hoc changes to a system and mitigates the risks of the changes degrading the system. A medical image analysis application developed using ProcessNet demonstrates that despite a large number of ad-hoc, incremental changes the system’s ability and accuracy in segmenting multiple anatomical regions in High Resolution Computed Tomography (HRCT) scans continue to improve.

[ Paper: PDF ]


ProcessNet made is possible for develop to manage a complex lung anatomy segmentation that segmented seven different anatomical structures building upon each other.

Incremental Engineering of Lung Segmentation Systems

Avishkar Misra, Arcot Sowmya, Paul Compton

Pacific Knowledge Acquisition Workshop (PKAW), as part of PacifAyman El-Baz (Editor), Jasjit S. Suri (Editor), Lung Imaging and Computer Aided Diagnosis, CRC Press, ISBN: 439845573, 2011.

Incremental and ad hoc development of medical image segmentation systems is an inevitable consequence of engineering systems, as the available data, expertise, and techniques evolve. The risk of a change degrading the system’s performance restricts the expert’s ability to build complex systems and confidently assimilate new data and techniques. In this chapter, an incremental validated change strategy called ProcessNet that mitigates these risks has been presented. Although ProcessNet may not eliminate all risks of degradation, the experiments support the idea that the accumulated set of cornerstone cases offers a data-driven means to assess the quality of a change to the system. This is the same argument and experience that underlies standard RDR.

ProcessNet captures and validates knowledge represented explicitly (such as in the form of rules) or implicitly in the libraries and algorithms defined in the source code of a process. The use of cornerstones at each process means that the knowledge behind heuristically defined algorithms can be supported by evidence grounded in data. The system developed using ProcessNet demonstrates that despite a large number of ad hoc revisions, a good solution can be discovered and improved on over time in a systematic manner. Although the ProcessNet framework has been applied specifically for the task of segmenting lungs and other anatomical structures in HRCT images, the framework is generic enough to address similar incremental ad hoc engineering for other types of medical image segmentation systems or computer vision systems in general.

[ Book Chapter: PDF ]