<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
	<channel>
		<title>Deep Learning on xiaopan&#39;s blog</title>
		<link>https://ixiaopan.github.io/blog/tags/deep-learning/</link>
		<description>Recent content in Deep Learning on xiaopan&#39;s blog</description>
		<generator>Hugo</generator>
		<language>en</language>
		
		
		
		
			<lastBuildDate>Wed, 24 Nov 2021 00:00:00 +0000</lastBuildDate>
		
			<atom:link href="https://ixiaopan.github.io/blog/tags/deep-learning/index.xml" rel="self" type="application/rss+xml" />
			<item>
				<title>Deep Learning - Weight Initialization</title>
				<link>https://ixiaopan.github.io/blog/post/dl/initilisation/</link>
				<pubDate>Wed, 24 Nov 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/initilisation/</guid>
				<description>&lt;p&gt;The goal of machine learning is to find the best parameters. To train the model, we must start from some values. So, what do the initial parameters look like? Can we set all of them to the same value? Or we assign random values to the weights? In this post, we will talk about another important aspect of learning — weight initialization.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Deep Learning - Regularization</title>
				<link>https://ixiaopan.github.io/blog/post/dl/regularization/</link>
				<pubDate>Mon, 22 Nov 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/regularization/</guid>
				<description>&lt;p&gt;In the previous post, we introduced how to initialize weights. However, weights can be very large or small (great variance in weights). For larger weights, a tiny change in data will lead to large variance. For smaller weights, it has little influence on the model, which can be totally discarded.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Literature Review on NER</title>
				<link>https://ixiaopan.github.io/blog/post/dl/lr-ner/</link>
				<pubDate>Wed, 17 Nov 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/lr-ner/</guid>
				<description>&lt;p&gt;Named entity recognition (NER) is a classical task in NLP, which aims to identify meaningful chunks of text and classify them into appropriate entity types, such as PERSON or LOCATION. For instance, &amp;ldquo;Tom&amp;rdquo; is classified as &amp;ldquo;PERSON&amp;rdquo; while &amp;ldquo;England&amp;rdquo; is labelled as &amp;ldquo;LOCATION&amp;rdquo; in the sentence &amp;ldquo;Tom comes from England&amp;rdquo;. NER serves as the foundation for a range of downstream tasks such as information extraction. In practical applications, you may notice that some websites will autofill the application form from your resume. In this series of posts, we will explain NER in detail, from models, datasets to domain adaptation. First, let&amp;rsquo;s do a brief literature review on NER.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Deep Learning - RNN</title>
				<link>https://ixiaopan.github.io/blog/post/dl/rnn/</link>
				<pubDate>Sun, 26 Sep 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/rnn/</guid>
				<description>&lt;p&gt;When I first learned RNN, I found lots of articles about it. That&amp;rsquo;s good news, but most of them focus on the design of the architecture.The network itself is not difficult to understand, the real problem is why it works. What&amp;rsquo;s the intuition behind it? Why do we need &amp;ldquo;recurrence&amp;rdquo;? Unfortunately, few articles explain it clearly. Luckily, the book &lt;a href=&#34;https://www.amazon.co.uk/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618&#34;&gt;Deep Learning&lt;/a&gt; gives the answers.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Deep Learning - CNN</title>
				<link>https://ixiaopan.github.io/blog/post/dl/cnn/</link>
				<pubDate>Thu, 23 Sep 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/cnn/</guid>
				<description>&lt;p&gt;Convolutional Neural Networks (CNN) are widely used for image classification, object detection and other tasks related to images or videos in the field of computer vision. Plenty of architectures based on convolution operation have been proposed in recent years, such as AlexNet and ResNet. So, how do CNNs work?  Why not use MLP? In this post, we will go through the fundamentals of CNN and several ConvNets to develop a big picture of CNN.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Deep Learning - Neural Networks</title>
				<link>https://ixiaopan.github.io/blog/post/dl/architecture/</link>
				<pubDate>Sat, 18 Sep 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/architecture/</guid>
				<description>&lt;h2 id=&#34;neural-networks&#34;&gt;Neural Networks&lt;/h2&gt;&#xA;&lt;p&gt;Neural networks are simply function compositions.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Feedforward networks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;$$&#xA;y = f(g(x, \theta_g), \theta_f)&#xA;$$&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Recurrent networks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;$$&#xA;y_t = f(y_{t-1}, x_t, \theta)&#xA;$$&lt;/p&gt;&#xA;&lt;p&gt;Generally, you can think of neural networks as function approximation machines, where we want to find appropriate $\theta$ to make $f^{*}$ as similar as possible to the true function $f$.&lt;/p&gt;&#xA;&lt;p&gt;In practice, there are lots of things to consider when designing a neural network. Basically, we need to consider the following factors,&lt;/p&gt;</description>
			</item>
			<item>
				<title>Deep Learning - Preliminary</title>
				<link>https://ixiaopan.github.io/blog/post/dl/pre/</link>
				<pubDate>Thu, 09 Sep 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/pre/</guid>
				<description>&lt;p&gt;So far, we&amp;rsquo;ve covered most of the things that we should know about machine learning, including concepts, optimization, and popular models under the hood. Yet, some advanced techniques, such as the Gaussian Process and MCMC, are not mentioned. We will talk about them later. From now on, we will move to Deep Learning. But before that, we are going to revisit some math knowledge.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Deep Learning - PyTorch</title>
				<link>https://ixiaopan.github.io/blog/post/dl/pytorch/</link>
				<pubDate>Mon, 19 Jul 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/pytorch/</guid>
				<description>&lt;p&gt;The most popular deep learning frameworks nowadays are Tensorflow and PyTorch. Well, during my study, I use PyTorch more often. Recently, I am building the classic BiLSTM-CRF model using PyTorch. It&amp;rsquo;s a bit hard for me when operating matrices since it uses various advanced techniques about indexing and slicing. I think it&amp;rsquo;s necessary to explain these amazing functions. Therefore, I am going to write this post to revisit the most important aspects of PyTorch for future references.&lt;/p&gt;</description>
			</item>
			<item>
				<title>NLP - Text Representation</title>
				<link>https://ixiaopan.github.io/blog/post/dl/text-representation/</link>
				<pubDate>Sat, 10 Jul 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/text-representation/</guid>
				<description>&lt;p&gt;In the last post, we talked about text preprocessing techniques. However, even the data is clean now, they are still text. We still haven&amp;rsquo;t answered the question: how to covert text into numbers? In NLP parlance, this is called &lt;strong&gt;text representation&lt;/strong&gt;.&lt;/p&gt;</description>
			</item>
			<item>
				<title>NLP - Text Preprocessing</title>
				<link>https://ixiaopan.github.io/blog/post/dl/text-preprocessing/</link>
				<pubDate>Tue, 22 Jun 2021 00:00:00 +0000</pubDate>
				<guid>https://ixiaopan.github.io/blog/post/dl/text-preprocessing/</guid>
				<description>&lt;p&gt;From now on, we will focus on a specific domain — Natural Language Processing(NLP), in part because my summer project is about Named Entities Recognition(NER). Therefore, I need to know some text preprocessing techniques and have a good understanding of the state-of-art NLP models, particularly BiLSTM + CRF. The very first step in NLP is text preprocessing, so I am going to start from here.&lt;/p&gt;</description>
			</item>
	</channel>
</rss>
