What is the Confusion Matrix?
A Confusion matrix is an N x N matrix used for evaluating the performance of a classification model, where N is the number of target classes. The matrix compares the actual target values with those predicted by the machine learning model. This gives us a holistic view of how well our classification model is performing and what kinds of errors it is making.
For a binary classification problem, we would have a 2 x 2 matrix as shown below with 4 values:-
—> Let’s start with an example confusion matrix for a binary classifier (though it can easily be extended to the case of more than two classes):
What can we learn from this matrix?
- There are two possible predicted classes: “yes” and “no”. If we were predicting the presence of a disease, for example, “yes” would mean they have the disease, and “no” would mean they don’t have the disease.
- The classifier made a total of 165 predictions (e.g., 165 patients were being tested for the presence of that disease).
- Out of those 165 cases, the classifier predicted “yes” 110 times, and “no” 55 times.
- In reality, 105 patients in the sample have the disease, and 60 patients do not.
Let’s now define the most basic terms, which are whole numbers (not rates):
- true positives (TP): These are cases in which we predicted yes (they have the disease), and they do have the disease.
- true negatives (TN): We predicted no, and they don’t have the disease.
- false positives (FP): We predicted yes, but they don’t actually have the disease. (Also known as a “Type I error.”)
- false negatives (FN): We predicted no, but they actually do have the disease. (Also known as a “Type II error.”)
Cyber Crime Use Case
Cybercrime is one of the Hot topics nowadays, We have to keep our software/app protected and secured from the hackers who intrude and destroy our system. Cyber Security is one of the necessities in today’s world.
Crime Development Over the Years
Some of the companies that use ML and Confusion Matrix to solve Cyber Crime challenges!
DEMISTO
Demisto’s security platform specializes in security orchestration, automation and response — SOAR for those in the know — to help larger companies and enterprises coordinate security threat response efforts. In addition to providing a visual dashboard where users can monitor all security alerts, Demisto uses machine learning to prioritize those alerts.
BLACKBERRY
BlackBerry, whose web-connected smartphones were once ubiquitous in certain circles, has pivoted and now sells software and services to big companies. Among the company’s specialties are cybersecurity solutions that employ AI and machine learning to prevent cybersecurity threats and automate clients’ threat response capabilities. In November 2018, BlackBerry acquired AI cybersecurity firm Cylance for $1.4 billion.
In the last ten years, the use of the Internet has been growing rapidly. However, as the Internet becomes part of daily activities, cybercrime is also increasing. According to the 2020 Cybersecurity Investment Report, by 2021, cybercrime will cost nearly US$6 trillion each year. For illegal activities, cybercriminals use any network computing device as the main means of communicating with the victim’s device, so the attacker can use the loopholes in the system to conduct propaganda and other activities from finances.
The association of security analysis and data analysis methods helps us analyze and classify crimes from integrated data (which may be structured or unstructured) from India. The main advantage of this work is the test analysis report, which can accurately classify crimes with 99% accuracy.
Precision vs. Recall
Precision is a useful metric in cases where False Positive is a higher concern than False Negatives.
Precision = TP / (TP + FP)Recall is a useful metric in cases where False Negative trumps False Positive.
Recall = TP / (TP + FN)Type-1 error is more dangerous than Type-2 error.
