Showing posts with label GSoC2014. Show all posts
Showing posts with label GSoC2014. Show all posts

Wednesday, June 29, 2016

SPREAD — System for Sharing and Publishing Research Data

We presented our work on MEDIator/SPREAD at SIIM 2016 in Portland, OR. 
 
In this study we developed a set of web-services that abstract the task of replicating and publishing subsets of data that are stored in curated biomedical data repositories. The overarching goal is to develop tools that can be configured against data repositories and give researchers, during the course of a research study, the ability to seamlessly share, with collaborators, data subsets; and upon conclusion of the study, publish the datasets for wider consumption.
 
The full paper can be found here.

Wednesday, August 12, 2015

[KDD 2015] MEDIator: A Data Sharing Synchronization Platform for Heterogeneous Medical Image Archives


One of my recent papers, "MEDIator: A Data Sharing Synchronization Platform for Heterogeneous Medical Image Archives" was presented at a KDD workshop this week in Sydney.

I could not present the paper myself at KDD, as my Australian visa was delayed. Luckily, my friend Denis who lives in Sydney helped me by presenting the paper. Much thanks to him!


Abstract
With the growing adaptation of pervasive computing into medical domain and increasingly open access to data, metadata stored in medical image archives and legacy data stores is shared and synchronized across multiple devices of data consumers. While many medical image sources provide APIs for public access, an architecture that orchestrates an effective sharing and synchronization of metadata across multiple users, from different storage media and data sources, is still lacking. This paper presents MEDIator, a data sharing and synchronization middleware platform for heterogeneous medical image archives. MEDIator allows sharing pointers to medical data efficiently, while letting the consumers manipulate the pointers without modifying the raw medical data. MEDIator has been implemented for multiple data sources, including Amazon S3, The Cancer Imaging Archive (TCIA), caMicroscope , and metadata from CSV files for cancer images.

Wednesday, October 8, 2014

Dimensions of Time

The summer in Stockholm was long and pleasant
I was reading an interesting study on Erasmus exchange programs. It states, "Not only do former Erasmus students have better employability skills than 70% of all students. What’s more, 27% of Erasmus alumni met their current life partner during their stay abroad, according to the recent Erasmus Impact Study." Go ahead and read the full report. ;) Thanks to the Erasmus Mundus, I have called Lisbon and Stockholm my homes. Of course, the definition of home has become pretty vague lately. The memories I had for the last two years were remarkably interesting.

Time is an interesting element. When we experience the same location after a long time, we feel like travelled through time. Today, the moment when I sit down at FA1, Instituto Superior Técnico after a long time (after around 1.5 years), for a lecture, remembering the legendary moments of EMDC (European Master in Distributed Computing) and Virtual Execution Module. I am sure I will feel the same when I return to Sweden/KTH for the second part of my PhD.

Most of the airports currently provide free Internet for a limited time, such as an hour per device. Since I usually have 3 devices with me during the travels (a laptop, a tab, and a phone), I always get 3x time. :D My devices remember the wifi networks that they have connected to. It is always interesting to view them - they are from universities, airports, hostels, hotels, homes/apartments in Colombo, Lisbon, and Stockholm, and cafes with Internet! They are the logs of the places that I have visited, the lands that host my homes, and my journeys between the destinations. Wish everyone offer unlimited Internet.

Today we got an update regarding the schools participated in Google Summer of Code 2014 (GSoC 2014), from Google. "For the first time in seven years a new school has claimed the top spot—congratulations to International Institute of Information Technology - Hyderabad."

Time has its own dimensions. It is interesting to see how many things change in life and technology. I am sure that the world is becoming a better place to live. I hope I am not being over-optimistic. :)

Tuesday, October 7, 2014

Data Sharing Synchronization System with Infinispan

The below presentation describes part of the data sharing and synchronization system that I developed during the summer. This presentation focuses mostly on the medical data and images, along with the cancer imaging archive (TCIA) as the data source. The latter part of the summer was spent, extending the tool for the other data sources, such as Amazon S3, CA Microscope, or even meta data stored in a CSV file.
  

Friday, August 22, 2014

[GSoC 2014] Data Replication and Synchronization Tool

I have just completed my third summer of code as a student (and sixth summer of code, if I count all my involvement in the Google Summer of Code, including the 3 years as a mentor with AbiWord, mentoring 2 projects in 2011 and 2012, and in the pool of mentors in 2013). 

The code samples can be downloaded here. The work this summer was interesting and rewarding, and I loved the learning experience as much as I did with my first summer of code with AbiWord and second with OGSA-DAI/OMII-UK. Special thanks to my Google Summer of Code 2014 mentor Prof. Ashish Sharma from Emory University, Biomedical Informatics.

Thursday, May 29, 2014

Fault-tolerant data replication and synchronization with Infinispan

Figure 1. Deployment
Fault-tolerance
Having multiple instances running over different nodes provide fault-tolerance, as when one node terminates, the other nodes have the backup replica of the partitions stored in the terminated node. Figure 1 shows the higher level deployment view of the solution.

Design
Two distributed cache instances exist in InfDataAccessIntegration.
    protected static Cache userReplicasMap;
    protected static Cache replicaSetsMap;
userReplicasMap is a mapping of userId -> Array of replicaSetIDs. UserID could be the logged in user name. (for now, testing with random strings).
replicaSetsMap is a mapping of replicaSetID -> replicaSet
Figure 2. Core class hierarchy

Though this could be replaced with a single cache instance with the mapping of userID -> replicaSets, I decided to go with this design, as having two cache instances will be more efficient during searches, duplicates, and push changes. Hence, I decided to go with two cache instances design.

InfDataAccessIntegration provides the API for publisher/consumer, TCIAInvoker (which extends InterfaceManager, an abstract class I created) implements the TCIA integration to invoke these methods. Figure 2 provides a core class hierarchy of the system.
 
Figure 3. Execution Flow
Execution Flow
The execution flow is depicted by Figure 3.
* User logs in -> logIn() checks whether the user has already stored replicaSets from the Infinispan distributed Cache. If so, execute them all again. This would be changed later as we do not have to execute all. Rather, we need to execute for the diffs.

* The user performs new searches, for the images, series, collections, and the other meta data. New searches will create and write the replicaSet to the distributed cache, before returning the results.

The replicaSet for the image will be as,
TCIAConstants.IMAGE_TAG + "getImage?SeriesInstanceUID=" + seriesInstanceUID

For other information (meta data), such as collections, series, etc,
TCIAConstants.META_TAG + query;
Here, query is something like, "getSeries?format=" + format +
                "&Collection=" + collection +
                "&PatientID=" + patientID +
                "&StudyInstanceUID=" + studyInstanceUID +
                "&Modality=" + modality;
When a new instance starts now, and invokes the log in action for the same user, it will execute the queries for the stored replicaSets again, and reproduce the same results.

Further updates will be posted, when they are available. :-)

Tuesday, May 20, 2014

Summer in Lisboa.. ^_^

After some sunny days, it has started to rain heavily again in Lisboa. Coding period began yesterday, and I have committed some interesting bits to the project. Please have a look at the status update presentation below. Wait for more updates. :)

Wednesday, May 14, 2014

Initial moments with the Data Replication System

Wednesday, May 7, 2014

Publish/Consumer API for the Data Replication - Synchronization Tool


Monday, April 28, 2014

Design decisions of the TCIA data replication and synchronization tool

The initial presentation on the design alternatives and the design decisions of the new TCIA data replication - synchronization tool.

Monday, April 21, 2014

[GSoC 2014] Data Replication / Synchronization Tools


Project Mentor: Ashish Sharma
Short description: Consumers download the data by searching the image repository using the browser. The information that the consumer is interested in, gets updated whenever the data producers update or add patient information. The current download tool lacks the ability to track the relevant updates to the consumer. A pub-sub solution based on Apache CXF, utilizing the JAX-RS REST API of CXF will assist automated downloads to the consumers.


I am happy to be back wearing the student hat in the GSoC after a long time, with the transition of Student (AbiWord) -> Student (OMII-UK) -> Mentor (AbiWord) -> Mentor (AbiWord) -> Mentor (AbiWord) -> Student (BMI-CCI). Interestingly, this is the 10th GSoC, and the 6th Google Summer of Code that I am involved in. This is also my 3rd time as a student, and the first time ending up in the de-duplication, with Freenet and Emory BMI, where both of them are two awesome project communities. My thanks goes to the mentors and developers from both organizations. Hope I will be able to work with Freenet in the upcoming years, as I could work with only one organization in a summer. I have already started playing with Freenet, as a side effect of the GSoC. :-)

I will probably keep this blog updated with information related to the GSoC 2014. My sincere thanks goes to my supervisor Prof. Luis Veiga for his continuous motivation, and also for encouraging me in GSoC. My thanks also goes to OGSA-DAI mentors and developers from the EPCC at the Edinburgh University, which was the first university affiliated lab that I worked with, for a GSoC. I loved that experience. I loved being with AbiWord as a student and as a mentor. This year, AbiWord chose not to participate in the GSoC, after its successful participation since 2006 to 2013.