Blog posts

Transformer Network in Pytorch from scratch

8 minute read

Published: June 22, 2021

Step by step implementation of “Attention is all you need” with animated explanations.

This is a supplementary post to the medium article Transformers in Cheminformatics.

Named entity recognition with simple Attention

less than 1 minute read

Published: November 10, 2020

NER implementation hosted within browser using Tensorflow-JS.

Definition from Wikipedia

Named Entity Recognition is a subtask of information extraction that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, etc. See demo below. Continue reading for model explanation and code.

Encoder - Attention - Decoder

10 minute read

Published: November 10, 2020

Explaining Attention Network in Encoder-Decoder setting using Recurrent Neural Networks

Encoder-Decoder paradigm has become extremely popular in deep learning particularly in the space of natural language processing. Attention modules complement encoder-decoder architecture to make learning more close to humans way. I present a gentle introduction to encode-attend-decode. I provide motivation for each block and explain the math governing the model. Further, I break down the code into digestible bits for each mathematical equation. While there are good explanations to attention mechanism for machine translation task, I will try to explain the same for a sequence tagging task (Named Entity Recognition).

Encode-Attend-Decode Architecture

In the next part of the series, I will use the architecture explained here to solve the problem of Named Entity Recognition

Seq2Seq Machine Translation

less than 1 minute read

Published: November 08, 2020

I explore Seq2Seq model in Pytorch to build a neural machine translation system. Currently the system translates from German to English. In this series, I will explore various state-of-the-art NLP architectures to build NMT systems algorithms and hope to focus on English to Hindi translation. I will also attempt to provide simplified mathematical explanations of the models as well as implementation details.

Digit Classification

less than 1 minute read

Published: November 01, 2020

Digit Recognition using Deep Learning

Webapp to recognize handwritten digits between 0 and 9. Model trained using Keras and served using Tensorflow.js