K-Nearest Neighbors(KNN) is a distance-based algorithm in machine learning used for both classification and regression. It’s simple and intutive to understand because it doesn’t require extra training step and the idea behind it is simple enough — similar points are tends to be close to each other. In this article, we will learn how KNN works.

I remembered that I learned a little stuff about information when I was in my undergraduate course. Well, maybe I was wrong. It’s been almost ten years. But when it comes to information theory, I know that we will talk about a person — Shannon. So what’s the information theory? What problems does it help to solve?

When I first learned machine learning, I was scared by the complicated formulas. I spent much time going over subjects like Linear Algebra and Calculus since I’d already forgotten them. But with time, I feel more and more confident in understanding them, though they sometimes still confuse me. Anyway, in my opinion, there is no need to know every detail about each equation, after all, we are not mathematicians. Instead, learning how to use these math formulas to solve real-world problems is the key.

The goal of machine learning is to infer an unknown pattern from data. However, both parameters and predictions are estimated values, so how confident are we in these values? To measure uncertainty, we use probability. On the other hand, Bayes' theorem provides a framework for us to invert the problem into a forward process, where we observe data from parameters instead of making inferences from data. Probability Random Variable At the beginning, let’s have a quick refresh on probability.

In semester 2, I took a module called Semantic Web Technologies. There are some reasons why I choose this module. At first glance, the name itself sounds not appealing. Actually, it does. But it’s still on my shortlist because I’ve been working on the web for many years and I wondered what the semantic web was. On the other hand, I was not interested in modules like Advanced Databases since I’ve learnt database during my undergraduate degree in Computer Science.