PyTorch early release version was announced yesterday 1/19. PyTorch is currently maintained by Adam Paszke, Sam Gross and Soumith Chintala. The first question that comes to mind is What exactly is PyTorch? Well to put in the words of the makers, PyTorch gives
GPU Tensors, Dynamic Neural Networks and deep Python integration.
It’s a Python first library, unlike others it doesn’t work like C-Extensions, with a minimal framework overhead, integrating with acceleration libraries such as Intel MKL and NVIDIA (CuDNN, NCCL) to maximise speed.
Let’s take a pause here and try to realise that till last few months, people were under the assumption that the deep learning library ecosystem was stabilising but it was far from the ground reality. Cutting edge tech in that ecosystem is ensuring efficient support for dynamic computation graphs and PyTorch just aces that is all aspects.
Dynamic computation graphs arise whenever the amount of work that needs to be done is variable. This may be when we’re processing text, one example being a few words while another being paragraphs of text, or when we are performing operations against a tree structure of variable size. This problem is particularly prominent in particular subfields, such as natural language processing, where I spend most of my time.
PyTorch is heavily influenced by Chainer and DyNet. In Chainer’s words, it is a difference between “Define-and-Run” frameworks and “Define-by-Run” frameworks. TensorFlow is a “Define-and-Run” framework where one would define conditions and iterations in the graph structure whereas in comparison Chainer, DyNet, PyTorch are all “Define-by-Run” frameworks. In this case at runtime the system generates the graph structure. This is closer to writing code in any language as a for loop in code will behave as a for loop inside the graph structure as well. TensorFlow doesn’t handle dynamic graphs very well though there are some not so flexible and frankly quite limiting primitive dynamic constructs.
Do follow me on twitter and you can also signup for a small and infrequent mailing list that I maintain. If you want to understand Deep Learning, go through this Medium post.
Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising &sponsorship opportunities.
To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.
If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!