courtesy of FreePIk

The current NY Times article on “Lawsuits Take the Lead in Fight Against Disinformation” makes me think its about time to enforce protections for certain kinds of speech in our democratic society. The article in question covers the legal fallout against abusive, anti-democratic claims of voter fraud, in a spate of cases by firms that were harmed by such accusations. But current legal remedies are limited. Why should we have to wait until after the damage is done for false information to be acted upon?

Besides the political debacle we just experienced around the past presidential election, this strikes a…


Making Sense of Big Data

PCA can be used to reject cyclic time-series behavior, and this works for anomaly detection.

Top: Noisy periodic signals, Bottom: Reduced anomaly signal. (All images by author).

Detecting an anomaly typically means thresholding a signal, to alarm when the signal is out-of-range. For something like an assembly line, where tolerances are precise, the difference between normal and abnormal is clear. But network traffic has a noisome characteristic that makes this hard. It varies with large daily cycles as customers’ activity peaks and wanes. One could try to tenderly tease out this daily variation by modelling it. However this trick using Principal Component Analysis (PCA) avoids that hard work.

The periodic components embedded in a set of concurrent time-series can be isolated by Principal Component Analysis (PCA), to…


Six principles that organize what I’ve learned over the years building models.

Source: Climate Change Risk Analysis for Projects in Kenya and Nepal  Van der Vat, M.P., J.E. Hunink, D. Stuparu
Source: Climate Change Risk Analysis for Projects in Kenya and Nepal  Van der Vat, M.P., J.E. Hunink, D. Stuparu
Photo credit: Climate Change Risk Analysis for Projects in Kenya and Nepal

Decision Quality systematizes the best practices for modeling decisions, which are practices that the field of Data Science sorely needs to adopt.¹

Introduction

Let’s say you are a Data Scientist at your first meeting with your client. Or you are a software engineering or program manager thinking of starting a project where you can apply what you know of Machine Learning (ML) or Artificial Intelligence (AI). Perhaps analysts on your team suggested a project based on their preliminary data analysis and the promise of almost unlimited data. …

John Mark Agosta

"Data Science" is a broadly encompassing term, and I focus on modelling, specifically the initial statistical formulation stages.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store