This is an experience report giving challenges while building automated , AI empowered application that converts the mundane job of an IT recruiter to an augmented job.

Problem: Recruiters hated transcribing resume data into long forms to enter into our ATS (Applicant Tracking System) . Using standard resume parsers didn't help because these parsers could not understand most of the technical words in the resumes. We tried buiding our own parser using the GATE library. It worked but GATE is single threaded and runs into memory issues at scale. Also training the parser to recognize thousands of technical words would need us to invest in a large team of professionals. We had to make this work at web scale so that many users could train the parser to recognize technical terms that were common to each user's context.

The UX has to be as smooth as a prosthetic extension. The user experience should get the user dependent on and addicted to using the application. Performance assumes central role - latency of a few milliseconds can send users off to other distractions. Building user retention and loyalty is even more challenging. This report shows how the principles of "Game Thinking" can be used to engage and motivate the users to become trainers of ML/NLP algorithms.

Solution: We built a highly scalable containerized microservice to parse resumes using GATE. We deployed these Docker containers on AWS and used Kubernetes to do auto scaling which led to an amazingly fast resume parsing service which could parse a hundred resumes in less than a minute. We worked around the memory leak issue by implementing the circuit breaker pattern.

According to Raph Koster users derive pleasure by learning to use your application. We built the application to "teach" the users to "train" the parser to recognize terms that are relevant to their specific context. This is supposed to give the users a pleasurable experience and a feeling of empowerment.We will present how the progression of hypothesis-driven sprints is helping us get closer to the goal.

 
88 favorite thumb_down thumb_up 3 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

This is a 45-minute talk divided into 4

1) Introduction- What is an augmented job- 5 minutes

2) Game thinking principles - 5 minutes

3) Cloud native, scalable, containerized microservice - 15 minutes

4) Training the NLP/ML algorithms - 15 Minutes

4) Q & A - 5 Minutes

Would it make sense to reduce the time talking about the solution and actually demonstrating it? Would it make sense to do a code walk through? I suspect that both these activities need more time and would break the flow of the presentation. Also code walkthrough may not be well understood by everyone. Instead I can provide links to articles to those who want to do a deep dive. Here are 2 such links

https://vinayakjoglekar.wordpress.com/2014/10/03/resume-ranking-using-machine-learning-implementation/

https://vinayakjoglekar.wordpress.com/2014/06/24/ranking-resumes-using-machine-learning/

Learning Outcome

Developers will leave the talk with a better understanding of design of microservices. Data Scientists will wrap their heads around leveraging engaging UX to get users to build training sets- an idea that can be used to build any training set.

Target Audience

This is targeted for Developers, Lean UX designers and Data Scientists. Executives and entrepreneurs will find it interesting too.

Prerequisite

The participant should be familiar with basic concepts of AWS, Docker, K8s, microservices , machine learning and natural language processing.

schedule Submitted 1 month ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Naresh Jain
    By Naresh Jain  ~  1 month ago
    reply Reply

    Thanks for the proposal, Vinayak. This proposal appears to be more beginner level. We've covered beginner level topics for a few years now. We are looking at more advanced topics for 2018. One suggestion would be is to convert this into a case study and share some of the challenges you faced and how your team overcame those challenges. Thanks.

    • Vinayak Joglekar
      By Vinayak Joglekar  ~  3 weeks ago
      reply Reply

      Naresh,

      Thanks for your quick response. I am putting together a talk that would highlight some of the technical and non technical challenges faced by us while building an engaging UX. It needs some prior knowledge of high level concepts of ML , NLP and Microservices. I hope that it is sufficiently "Advanced" to be meaningful for the delegates. I will be modifying my present submission as suggested by you.

      Thanks,

      Vinayak

      • Naresh Jain
        By Naresh Jain  ~  3 weeks ago
        reply Reply

        Thanks, Vinayak. Looking forward to it.