Neural Network Can Diagnose Covid-19 from Chest X-Rays

• New study is 98.4% accurate at detecting Covid-19 from X-rays.
• Researchers trained a convolutional neural network on Kaggle dataset.
• The hope is that the technology can be used to quickly and effectively identify Covid-19 patients.

As the Covid-19 pandemic continues to evolve, there is a pressing need for a faster diagnostic system. Testing kit shortages, virus mutations, and soaring numbers of cases have overwhelmed health care systems worldwide. Even when a good testing policy is in place, lab testing is arduous, expensive, and time consuming. Cheap antigen tests, which can give results in 30 seconds, are widely available but suffer from low sensitivity; The tests correctly identifying just 75% of Covid-19 cases a week after symptoms start [2].

Shashwat Sanket and colleagues set out to find an easy, fast, and accurate alternative using simple chest X-ray images. The team found that bilateral changes seen in chest X-rays of patients with Covid-19 can be analyzed and classified without a radiologist’s interpretation, using Convolutional Neural Networks (CNNs). The study, published in the September issue of Multimedia tools and Applications, successfully trained a CNN to accurately diagnose Covid-19 from Chest X-Rays, achieving an impressive 98.4% classification accuracy.. The journal article, titled Detection of novel coronavirus from chest X-rays using deep convolutional neural networks, shows some exciting promise in the ongoing efforts to find ways to detect Covid-19 quickly and effectively, 

What are Convolutional Neural Networks?

A convolutional neural network (CNN) is a Deep Learning algorithm that resembles the response of neurons in the visual cortex. The algorithm takes an input image and weighs the relative importance of various aspects in the image. The neurons overlap to span the entire field of vision, comprising a completely connected network where neurons in one layer link to neurons in other layers. The multilayered CNN includes an input layer, an output layer, and several hidden layers. A simple process called pooling keeps the most important features while reducing the dimensionality of the feature map. 

One major advantage of CNNs is that, compared to other classification algorithms, the required pre-processing is much lower. In addition, CNNs use regularized weights over fewer parameters. This avoids the exploding gradient and vanishing gradient problems of traditional neural networks during backpropagation.

Data Prep

The study began with a Kaggle dataset containing radiography images. As well as chest X-ray images for 219 COVID-19 positive cases, the dataset also contained 1341 normal chest X-rays and 1345 viral pneumonia images. Random selection was used to reduce the normal and viral pneumonia images to a balanced 219 each. The model, which the authors dubbed CovCNNl, was trained with augmented chest X-ray images; The raw images were standardized with each other using transformations like shearing, shifting and rotation. They were also converted to the same size: 224 × 224 × 3 pixels. Following the augmentation, the dataset was split into 525 images for training and 132 images for testing. The following image, from the study authors, demonstrates how the augmented images appear. Image a in the top row shows how Covid-19 appears on an x-ray, in comparison to four normal chest X-rays:

Seven existing pre-trained transfer learning models were used in the study, including ResNet-101 (a 101 layers deep CNN), Xception (71 layers deep), and VGG-16, which is widely used in image classification problems but painfully slow to train [3]. Transfer learning takes lessons learned from previous classification problems and transfers that knowledge to a new task—in this case, correctly identifying COVID-19 patients.

Results

Four variant CovCNN models were tested for effectiveness with several metrics, including: accuracy, F1-score, sensitivity, and specificity. The F1 score is a combination of recall and precision; Sensitivity is the true positive rate—the proportion of correctly predicted positive cases; Specificity is the proportion of correctly identified negative cases. The CovCNN_4 model outperformed all the other models, achieving 98.48% accuracy, 100% sensitivity, and 97.73% specificity. This fine-tuned deep network contained 15 layers, stacked sequentially with increasing filter sizes. This next image shows the layout of the model:

The authors conclude that their covCNN_4 model can be employed to assist medical practitioners and radiologists with faster, more accurate Covid-19 diagnosis, as well as follow up cases. In addition, they recommend that their model’s accuracy can be further improved by “fusion of CNN and pre-trained model features”.

References

CNN Images: Adobe Creative Cloud

[1] Detection of novel coronavirus from chest X-rays using deep convolu…
[2] Fast coronavirus tests: what they can and can’t do
[3] VGG16 – Convolutional Network for Classification and Detection

Source Prolead brokers usa

How Businesses Are Using Data Analytics for Better Operational Efficiency

As the market gets more competitive with time, businesses are altering their strategies to sustain and cater to changing customer needs better. The present era customers have smartened up considerably! They know what they want, and luring them with glitzy ads and lofty marketing pitches does not cut much ice anymore. They want better value for money and an enhanced experience. So, businesses need to offer better service, enhance product quality, and become more productive and efficient. 

Data analytics is a big weapon for enhancing the operational efficacy of businesses.

Nowadays, businesses of varying types and sizes are resorting to data analytics applications to enhance efficiency and productivity levels. They obtain data from a number of sources- both offline and online. This huge amount of data is then compiled and analysed by using specialized BI solutions. The resultant reports and insights help the businesses to get a better grasp of various nuances of operations. They resort to using cutting-edge data analytics applications, including power bi solutions.

How using data analytics software and applications can be useful for businesses. 

• It helps businesses identify market needs- The BI and data analytics tools can be useful for identifying market needs. Data obtained from online and offline customer surveys, polls and other types of feedback are compiled and analysed by such applications. The results can help businesses understand the precise needs of the market. This can vary from one location to another. When businesses can understand regional market needs better, they can tweak their production plan accordingly. It proves to be beneficial in the long run. 

It aids the brands to detect and eliminate Supply Chain hurdles- For a brand manufacturing physical products, supply chain optimization can prove to be tedious. Logistics related issues can crop unexpectedly, hampering the sales and supply chain system. Issues that can affect the supply chain include shipping delays, damage to fragile items, whether caused by hassles, employee issues, etc. This is where data analytics tools like Power BI can come in handy. 

• The data collected through sensors, cloud services and wearable devices are analysed by such applications. The generated reports help power bi consultants figure out the existing loopholes leading to disruptions in the supply chain. They can thereafter come up with strategies to tackle and eliminate such issues.  

It helps identify and resolve Team-coordination issues- Sometimes, a company may find it hard to achieve its operational target owing to improper and inadequate sync between various departments. The departments like HR, sales and advertising may not have good sync with one another. This can lead to inefficient resource sharing. For the management, it may be hard to figure out these internal glitches. However, hiring a data analytics expert can be helpful in resolving such conditions.

• A veteran power bi developer can use the tool to analyze collected data and find out the issues leading to a lack of sync between various departments. Thereafter, suitable remedial measures can be taken to boost resource sharing, and that can help augment efficiency. 

• It helps detect employee and team productivity issues- Not everyone in a team in a company has equal efficacy and productivity. A senior team member and employee may work smarter and faster than newly inducted ones. Sometimes, disgruntled employees may deliberately work in an unproductive way. The overall output gets affected when there are such issues affecting the productivity and efficacy of the employees in a company. 

For the management, checking the efficacy of every single employee may not be easy. In a large-sized organization, it is near impossible. However, identifying employee efficacy and productivity becomes easier when a suitable data analytics solution is used. Hiring a power bi development professional can be handy in such situations. By identifying factors leading to employee productivity deficit, corrective measures can be deployed.

• It helps detect third-party/vendor related issues- In many companies, working with third-party vendors and suppliers becomes necessary. Businesses may rely on such vendors for the supply of raw materials, and they also hire such vendors to outsource specific operations. Sometimes, the operational output of the company may get affected owing to reliance on a vendor not suited for its needs. The suitability of such vendors can be understood well by deploying data analytics services.  

• It aids in understanding speed related issues- Sluggishness in production may affect the output in a business setup, for sure. Production or manufacturing involves a number of stages, and delay in one or more stages can affect productivity and efficacy. It may be hard for the company management to fathom what is causing the delay in the production workflow. The reasons can be worn out by machinery or unskilled workforce. Deploying the latest data analytics solutions can be useful for detecting and resolving the issues affecting production speed.  
• It helps in detecting IT infrastructure issues- Sometimes, your business may find it hard to achieve operational targets owing to the usage of outdated or ageing IT infrastructure. It is both hardware and software related issues that affect output and efficiency. The legacy systems used in some organizations bottleneck the prowess of a skilled and efficient workforce- as it has been seen. Deploying the latest data analytics solutions helps the companies understand which part of the IT infrastructure is causing the deficit in output.  

It aids in understanding cost overrun factors- In every company, incurring a cost is a prerequisite for keeping the workflow alive. However, it is also necessary that the running expenditure of the workplace is kept within a limit. It can be hard to figure out if the money spent after departments like electricity, internet, sanitation etc., are being kept within a limit or overspending is taking place. Sometimes, hidden costs may be involved, which may skip scrutiny of the accounts departments. 

• When data analytics tools are used, it is easier to find out instances of cost overrun in such setups. The management then can take up corrective measures to ensure running cost is kept within feasible limits.  

Summing it up

Usage of data analytics tools like Power BI helps a company in figuring out issues that are bottlenecking productivity and output. The advanced data analysis and report generation capabilities of such tools help businesses fathom issues that can be hard to interpret and analyze otherwise. By using such tools, businesses can also make near accurate predictions about market dynamics and customer preferences. However, to leverage the full potential of such tools, hiring suitable data analytics professionals will be necessary.

Source Prolead brokers usa

The Top Skills for a Career in Datascience in 2021

Datascience is exploding in popularity due to how it’s tethered to the future of technology, supply-demand for high paying jobs and being on the bleeding edge of corporate culture, startups and innovation!

Students from South and East Asia especially can fast track lucrative technology careers with data science even as tech startups are exploding in those areas with increased foreign funding. Think carefully. Would you consider becoming a Data Scientist? According to Coursera:

A data scientist might do the following tasks on a day-to-day basis:

• Find patterns and trends in datasets to uncover insights
• Create algorithms and data models to forecast outcomes
• Use machine learning techniques to improve quality of data or product offerings
• Communicate recommendations to other teams and senior staff
• Deploy data tools such as Python, R, SAS, or SQL in data analysis
• Stay on top of innovations in the data science field

In a data-based world of algorithms, data science encompasses many roles since data scientists help organizations to make the best out of their business data.

In many countries there’s still a shortage of expert data scientists that are familiar with the latest tools and technologies. As fields such as machine learning, AI, data analytics, cloud computing and related industries get moving, the labor shortage of skilled professionals will continue.

Some Data Science Tasks Are Being Automated with RPA

As some tasks of data scientists become automated, it’s important for programming students and data science enthusiasts to focus on learning hard skills that should continue to be in demand well into the 2020s and 2030s. As such I wanted to make an easy list of the top skills for knowledge workers in this exciting area of the labor market for tech jobs.

Shortage of Data Scientists Continues in Labor Pool

The reality is for most organizations in most places there’s still a significant shortage of data scientists.

So the idea here is to acquire skills that are more difficult for RPA and other automation technologies to automate at organizations. It’s also important to specialize in skills where business penetration is high but increasing faster as the majority of businesses are adopting the trend, like Cloud computing and Artificial Intelligence.

AI Jobs Will Grow Significantly in the 2020s

In India, according to LinkedIn, AI is one of the fastest growing jobs. LinkedIn notes, Artificial Intelligence roles play an important role in India’s emerging jobs landscape, as machine learning unlocks innovation and opportunities. Roles in the sector range from helping machines automate processes to teaching them to perceive the environment and make autonomous decisions. This technology is being developed across a range of sectors, from healthcare to cybersecurity.

The top skills they cite are Deep Learning, Machine Learning, Artificial Intelligence (AI), Natural Language Processing (NLP), TensorFlow.

With such a young cohort of Millennials and GenZ, countries like India and Nigeria are unique in the latter half of the 2020s and 2030s as being the most productive workforces in the world and, yes, demographics really matter here. So for a young Indian, Nigerian, Indonesian, Brazilian or Malaysian in 2021 this really is the right time to start a career in data science since that could lead to bigger and brighter things.

So let’s start the list of generic skills that I think matter the most for the future data scientists and students now studying programming and related fields of skills that are transferable to the innovation boom that is coming.

1. Machine Learning

Machine learning is basically a branch of artificial intelligence (AI), that has become one of the most important developments in data science. This skill focuses on building algorithms designed to find patterns in big data sets, improving their accuracy over time.

The more data a machine learning algorithm processes, the “smarter” it becomes, allowing for more accurate predictions. 

Data analysts (average U.S. salary of $67,500) aren’t generally expected to have a mastery of machine learning. But developing your machine learning skills could give you a competitive advantage and set you on a course for a future career as a data scientist.

2. Python

Python is often seen as the all-star for an entry into the data science domain. Python is the most popular programming language for data science. If you’re looking for a new job as a data scientist, you’ll find that Python is also required in most job postings for data science roles. 

Why is that?

Python libraries including Tensorflow, Scikit-learn, Pandas, Keras, Pytorch, and Numpy also appear in many data science job postings.

According to SlashData, there are 8.2 million active Python users with “a whopping 69% of machine learning developers and data scientists now using Python”.

Python syntax is easy to follow and write, which makes it a simple programming language to get started with and learn quickly. A lot of data scientists actually come from backgrounds in statistics, mathematics, or other technical fields and may not have as much coding experience when they enter the field of data science. Since BigData and AI are exploding, the Python community is of course as you know large, thriving, and welcoming. 

 A library in Python is a collection of modules with pre-built code to help with common tasks. The number of related libraries to Python is staggering to me.

You may want to familiarize yourself with what they actually do:

Data Cleaning, Analysis and Visualization

• NumPy: NumPy is a Python library that provides support for many mathematical tasks on large, multidimensional arrays and matrices.
• Matplotlib: This library provides simple ways to create static or interactive boxplots, scatterplots, line graphs, and bar charts. It’s useful for simplifying your data visualization tasks.
• Pandas: The Pandas library is one of the most popular and easy-to-use libraries available. It allows for easy manipulation of tabular data for data cleaning and data analysis.
• Scipy: Scipy is a library used for scientific computing that helps with linear algebra, optimization, and statistical tasks.

• Seaborn: Seaborn is another data visualization library built on top of Matplotlib that allows for visually appealing statistical graphs. It allows you to easily visualize beautiful confidence intervals, distributions and other graphs.
• Statsmodels: This statistical modeling library builds all of your statistical models and statistical tests including linear regression, generalized linear models, and time series analysis models.
• Requests: This is a useful library for scraping data from websites. It provides a user-friendly and responsive way to configure HTTP requests.

Then there are the Python libraries more related to machine learning itself.

• Tensorflow: Tensorflow is a high-level library for building neural networks. Since it was mostly written in C++, this library provides us with the simplicity of Python without sacrificing power and performance.
• Scikit-learn: This popular machine learning library is a one-stop-shop for all of your machine learning needs with support for both supervised and unsupervised tasks.
• Keras: Keras is a popular high-level API that acts as an interface for the Tensorflow library. It’s a tool for building neural networks using a Tensorflow backend that’s extremely user friendly and easy to get started with.
• Pytorch: Pytorch is another framework for deep learning created by Facebook’s AI research group. It provides more flexibility and speed than Keras.

So as you can see Python is a great foot-in-the-door skill that’s related to entering the field of data science.

3. R, A Great Programming Language for Data Science in Industry

R is not often mentioned necessarily with data science. Here’s why I think it’s important.

R is another programming language that’s widely used in the data science industry. One can learn data science with R via a reliable online course. R is suitable for extracting key statistics from a large chunk of data. Various industries use R for data science like healthcare, e-commerce, banking and others.

For example, a Harvard certificate in data science has a section on R.

R’s open interfaces allow it to integrate with other applications and systems. As a programming language, R provides objects, operators and functions that allow users to explore, model and visualize data.

As you may know, machine learning is entering the finance, banking, healthcare and E-commerce sectors more and more.

R is more specialized than Python and as such might have higher demand in some sectors. R is typically used in statistical computing. So if you are technically minded R could be a good bet because R for data science focuses on the language’s statistical and graphical uses. When you learn R for data science, you’ll learn how to use the language to perform statistical analyses and develop data visualizations. R’s statistical functions also make it easy to clean, import and analyze data. So if that’s your cup of tea, R is great for finance at the intersection of data science.

4. Tableau for Data Analytics

With more data comes the need for better data analytics. The evolution of data science workers really is a marvel to behold. In a sense data science is nothing new and is just the practical application of statistical techniques that have existed for a long time. But honestly I think data analytics, and more Big Data changes how we can visualize and use data to drive business outcomes.

Tableau is an in-demand data analytics and visualization tool used in the industry. Tableau offers visual dashboards to understand the insights quickly. It supports numerous data sources, thus offering flexibility to data scientists. Tableau offers an expansive visual BI and analytics platform and is widely regarded as the major player in the marketplace.

It’s worth taking a look at if data visualization interests you. Other data visualization tools might include PowerBI, Excel and others.

5. SQL and NoSQL

Even in 2021, SQL has a surprisingly common utility for data science jobs. SQL (Structured Query Language) is used for performing various operations on the data stored in the databases like updating records, deleting records, creating and modifying tables, views, etc. SQL is also the standard for the current big data platforms that use SQL as their key API for their relational databases.

So if you are into databases, the general operations of data, data analytics and working in a data-driven environment SQL is certainly good to know.

Are you good at trend spotting? Do you enjoy thinking critically with data? As data collection has increased exponentially, so has the need for people skilled at using and interacting with data to be able to think critically, and provide insights to make better decisions and optimize their businesses.

Becoming a data analyst could be more enjoyable than you think, even if it lacks some of the glamor and hype of other sectors of data science.

According to Coursera, data analysis is the process of gleaning insights from data to help inform better business decisions. The process of analyzing data typically moves through five iterative phases:

• Identify the data you want to analyze
• Collect the data
• Clean the data in preparation for analysis
• Analyze the data
• Interpret the results of the analysis

6. Microsoft PowerBI

With Azure doing so well in the Cloud, Microsoft’s PowerBI is good to specialize in if you are less interested in algorithms and more interested in data analytics and data visualization. So what is it?

Microsoft Power BI is essentially a collection of apps, software services, tools, and connectors that work together to work on our data sources to turn them into insights, visually attractive, and immersive reports. 

Power-Bi is an all-in-one high level tool for the data analytics part of data science. It can be thought of as less of a programming-language type application, but more of a high level application akin to something like Microsoft Excel.

If you are highly specialized in PowerBI it’s likely you’d always be able to find productive work. It’s what I would consider a safe bet in data science. While it’s considered user friendly, it’s not open source, which might put off some people.

7. Math and Statistics Foundations or Specialization

It seems only common sense to add this but if you are interested in a future with algorithms or deep learning, a background in Math or Statistics will be very helpful. Not all data scientists will want to go in this direction but the data scientist will be expected of course to understand the different approaches to statistics — including maximum likelihood estimators, distributors, and statistical tests — in order to help make recommendations and decisions. Calculus and linear algebra are both key as they’re both tied to machine learning algorithms. 

The easiest way to think of it is that Math and Stats are the building blocks of Machine Learning algorithms. For instance, statistics is used to process complex problems in the real world so that data scientists and analysts can look for meaningful trends and changes in data. In simple words, statistics can be used to derive meaningful insights from data by performing mathematical computations on it. Therefore the aspiring knowledge worker student of data science will want to be strong in Stats and Math. Since many algorithms will be dealing with predictive analytics, it will also be useful to be well-grounded in probability.

8. Data Wrangling

The manipulation of data or wrangling is also an important part of data science, e.g. data cleaning. Data manipulation and wrangling make take up a lot of time but ultimately help you in taking better data-driven decisions. Some of the data manipulation and wrangling generally applied is – missing value imputation, outlier treatment, correcting data types, scaling, and transformation. This in general makes Data Analysis possible.

Data wrangling is essentially the process of cleaning and unifying messy and complex data sets for easy access and analysis. With the amount of data and data sources rapidly growing and expanding, it is getting increasingly essential for large amounts of available data to be organized for analysis. There are specialized software platforms that specialize in the data analytics lifecycle.

The steps of this cycle might include:

• Collecting data: The first step is to decide which data you need, where to extract it from, and then, of course, to collect it (or scrape it).
• Exploratory data analysis: Carrying out an initial analysis helps summarize a dataset’s core features and defines its structure (or lack of one).
• Structuring the data: Most raw data is unstructured and text-heavy. You’ll need to parse your data (break it down into its syntactic components) and transform it into a more user-friendly format.
• Data cleaning: Once your data has some structure, it needs cleaning. This involves removing errors, duplicate values, unwanted outliers, and so on.
• Enriching: Next you’ll need to enhance your data, either by filling in missing values or by merging it with additional sources to accumulate additional data points.
• Validation: Then you’ll need to check that your data meets all your requirements and that you’ve properly carried out all the previous steps. This commonly involves using tools like Python.
• Storing the data: Finally, store and publish your data in a dedicated architecture, database, or warehouse so it is accessible to end users, whoever they might be.

Tools that might be used in Data wrangling are: Scrapy, Tableau, Parsehub, Microsoft Power Query, Talend, Alteryx APA Platform, Altair Monarch or so many others.

9. Machine Learning Methodology

Will data science become more automated? This is an interesting question. At its core, data science is a field of study that aims to use a scientific approach to extract meaning and insights from data. Machine learning, on the other hand, refers to a group of techniques used by data scientists that allow computers to learn from data. 

Machine learning are techniques that produce results that perform well without programming explicit rules. If data science is  the scientific approach to extracting meaning and insights from data, it is really a combination of information technology, modeling, and business management. However machine learning or even deep learning actually often does the heavy lifting.

Since there is just a massive explosion of big data, data scientists will be in high demand for likely the next couple of decades at least. Machine learning creates a useful model or program by autonomously testing many solutions against the available data and finding the best fit for the problem. Machine learning leads to deep learning and is the basis for artificial intelligence as we know it today. Deep learning is a type of machine learning, which is a subset of artificial intelligence.

So if a data scientist student is interested in working on AI, they will need a firm grounding in machine learning methodology. While machine learning requires less computing power, deep learning typically needs less ongoing human intervention. They are both being used to solve significant problems in smart cities and the future of humanity.

10. Soft Skills for Data Science

To work in technology soft skills can be huge differentiators when everyone on the team has the same level of knowledge. Communication, curiosity, critical thinking, storytelling, business acumen, product understanding and being a team player among many other soft skills are all important for the aspiring data scientist and these should not be neglected.

Ultimately data scientists work with data and insights to improve the human world. Soft skills are a huge asset for a programming student that wants to be a manager one day or even to transition to a more executive role later in life or become an entrepreneur after their engineering life is less dynamic. You will want to especially work on:

• Empathetic leadership skills
• Power of observation that leads to insight into others
• Good communication

Having more polished soft skills can also obviously enable you to perform better on important job interviews, in critical phases of projects and to have a solid reputation within a company. All of this greatly enhances your ability to move your career in data science forward or even work at some of the top companies in the world.

Conclusion

A career in data science is incredibly exciting when AI and Big Data permutate our lives more than ever before. There are many incredible resources online to learn about data science and particular career paths for programming, machine learning, data analysis and AI.

Finally whether you choose data science or machine learning will depend on your aptitude, interests and willingness to get post graduate degrees. They can be summarized by the following:

Skills Needed for Data Scientists

• Statistics
• Data mining and cleaning
• Data visualization
• Unstructured data management techniques
• Programming languages such as R and Python
• Understand SQL databases
• Use big data tools like Hadoop, Hive and Pig

Skills Needed for Machine Learning Engineers

• Computer science fundamentals
• Statistical modeling
• Data evaluation and modeling
• Understanding and application of algorithms
• Natural language processing
• Data architecture design
• Text representation techniques

I hope this has been a helpful introductory overview meant to stimulate students or aspiring students of programming, data science and machine learning while giving a sense of some key skills, concepts and software to become familiar with. The range of jobs in the field of data science is really quite astounding, all with slightly different salary expectations. The average salary for a data scientists in Canada (where I live) is $86,000, which is $5 million Indian Rupees (50 lakhs) for example.

Share this article with someone you know that might benefit from it. Thanks for reading. 

This article has 3,215 words. 

Source Prolead brokers usa

Interesting Ways Big Data Has Changed the Face of the eCommerce Industry

Big Data is trending right now, but how does it change the eCommerce industry? Let’s understand in detail. 

eCommerce is booming, and consumer’s data has become a lifeline for online stores. A huge volume of data is generated by the eCommerce industry when it comes to customer patterns and purchasing habits.

It is projected that by 2025, the digital universe of data will reach 175 zettabytes, a 61 percent increase. It includes e-commerce – tracking shoppers’ activities, their locations, web browser histories, and abandoned shopping carts. 

Modern tech such as Artificial intelligence (AI), Machine Learning, and Big Data is not just for books and sci-fi movies anymore. These are now one of the most common tools used in an E-commerce site’s performance optimization. 

Gartner reported that by 2020, 85% of customer communications might not require human intervention due to advancements in AI. Online businesses should have access to a large volume of data, enabling them to make better decisions about their customers, the products they recommend, and how they will plan and implement their marketing campaigns.

A great deal of success in e-commerce relies on Big Data to plan future business moves. Now before discussing how Big Data impacts eCommerce, let’s understand the meaning of Big Data Analytics. 

What Stands for Big Data Analytics

Source: FinanceOnline.com 

Big Data Analytics means examining a huge volume of data to identify hidden patterns, correlations, and other valuable insights. This enables online stores to make informed decisions based on data. 

E-commerce companies use Big Data analytics to understand their customers better, forecast consumer behaviour patterns, and increase revenue. According to the study conducted by BARC, some benefits brands can avail using Big Data: 

• Making data-driven decisions
• Improved control on business operations
• Deliver top-notch customer experience 
• Reduce operational cost 
• Allow customers to make secure online payments 
• Supply management and logistics 

The eCommerce market is skyrocketing, a source taken from Elluminatiinc.com, today, 2.15 billion people shop online, and this figure will continue to grow because customers today value comfort over anything else. 

Now think from the eCommerce business owner’s point of view, how they identify preferences of these billions of customers and provide them with a personalized experience. Here Big Data comes to the rescue. eCommerce Big Data includes structured and unstructured information about customers, such as their addresses, zip codes, shopping cart contents, and more.

Now think from the eCommerce business owner’s point of view, how they identify preferences of these billions of customers and provide them with a personalized experience. (start) When it comes to serving a huge customer range especially for multi-channel selling firms, it’s hard to manage and update constantly among sales channels. This can cause great damage to keep your loyal customers. That’s why, integrating your business into a reliable tool is the best way to deal with the problem. Here, BigData and LitCommerce comes to the rescue.

Email, video, tweets, and comments in social media are unstructured eCommerce parts that can also serve as valuable sources of information. The ability to examine shopping carts or show individual content based on an IP address via a content management system is already available to online retailers, but Big Data discovery will extend their capabilities in the short term.

Big Data in eCommerce

Are you able to benefit from this? Well, Big Data lets you organize information in a pretty structured way so that you can provide a top-notch experience to customers. As a result, e-commerce business owners gain valuable insight into the choices and behaviours of their customers, resulting in increased sales and traffic. 

The following are the most notable ways Big Data will affect eCommerce in the future. 

Enhance Customer Service

Big Data plays an important role in delivering excellent customer service because it keeps track of your existing and new customers’ records and helps you study their preferences to boost the engagement ratio. 

This process includes what your customers like, which payment method they follow, what kind of products they buy frequently, and much more. Consequently, eCommerce business owners understand the user’s mind and offer a personalized experience to drive sales and traffic. 

Here you can take an example of an online streaming service, Netflix. Along with personalization, it also has implemented autoscaling support to meet evolving needs of customers.  

Improve Payment Methods and Security

Basically, unsecured payments and a lack of variety in payment methods contribute to abandoned carts. For instance, customers would not purchase from your store if they don’t find their desired payment methods. eCommerce stores can improve conversion rates by offering a variety of digital payment methods, but it should be swift and secure. 

Big Data can also improve payment security in the future. A variety of payment methods and safe and secure transactions are important for customers. Here Big Data can detect fraud activity and ensure an amazing experience for customers. 

Business owners can set up alerts for transactions on the same credit card that are out of the ordinary or for orders coming from the same IP address using various payment methods.

Customised Offers 

Who does not like to get huge discounts on products they love? Of course, we all, right? Utilize customer data to determine specific offers relevant to their previous purchases and send them discount codes and other offers based on their buying habits.

Additionally, Big Data can be used to find potential customers who are browsing a website, abandon a purchase, or but don’t buy. You can send a customer an email inviting them to purchase a product they looked at or reminding them of it. Here you can see how Amazon and eBay perfected the art of online selling.

Helps to Conduct A/B Testing

To ensure a seamless and efficient online experience, A/B testing is essential. Detecting bugs and removing errors will help your business grow. In addition to testing your pricing model on a time-based basis with data collected from your store, the data collected will help you optimize the overall store performance.

Especially during days when the demand is high, incorrect pricing can make your retailers lose money. Additionally, marketers can identify where the lift occurs and how they can use it to increase volume in time-based A/B testing, further assisting them in determining discounting strategies.

Forecast Trends and Demand

Predicting trends and demand is equally important to meeting buyer’s needs. Having the right inventory on hand for the future is crucial for e-commerce. eCommerce brands can make plans for upcoming events, seasonal changes, or emerging trends via Big Data.

Businesses that sell products online amass massive datasets. Analysis of previous data allows them to plan inventory, predict peak times, forecast demand, and streamline operations overall.

E-commerce companies can also offer various discounts to optimize pricing. Machine Learning and Big Data here make it easier to predict when and how discounts should be offered as well as how long they should last.

Big Data is Here to Stay for a Long

The above points clearly define that Big Data is making eCommerce better. eCommerce is already being affected by Big Data, and that impact will only continue to grow. With its use, online stores do business faster and easier than ever before. Their business activities are improved through the use of Big Data in all areas, and customer satisfaction is always maintained.

Source Prolead brokers usa

Are Data Scientists Becoming Obsolete?

This question is raised on occasion. Salaries are not increasing as fast as they used to, though this is natural for any discipline reaching some maturity. Some job seekers claim it is not that easy anymore to find a job as a data scientist. Some employers have complained about the costs associated with a data science team, and ROI expectations not being met. And some employees, especially those with a PhD, complained that the job can be boring.

I believe there is some truth to all of this, but my opinion is more nuanced. Data scientist is a too generic keyword, and many times not even related to science. I myself, about 20 years ago, experienced some disillusion about my job title as a statistician. There were so many promising paths, but the statistical community, in part because of the major statistical associations and academic training back then, missed some big opportunities, focusing more and more on narrow areas such as epidemiology or census data, but failing to catch on serious programming (besides SAS and R) and algorithms. I was back then working on digital image processing, and I saw the field of statistics missing the machine learning opportunity and operations research in particular. I eventually called myself a computational statistician: that’s what I was doing, and it was getting more and more different from what my peers were doing. I am sure by now, statistics curricula have caught up, and include more machine learning and programming.

More recently, I called myself data scientist, but today, I think it does not represent well what I do. Computational or algorithmic data scientist would be a much better description. And I think this applies to many data scientists. Some, focusing more on the data aspects, could call themselves data science engineers or data science architects. Some may find the word business data scientist more appropriate. Junior ones are probably better defined as analysts.

Some progress has been made in the last 5 years for sure. Applicants are better trained, hiring managers are more knowledgeable about the field and have more clear requirements, and applicants have a better idea as to whether an advertised position is as interesting as it sounds in the description. Indeed, many jobs are filled without even posting a job ad, by directly contacting potential candidates that the hiring manager is familiar with, even if by word-of-mouth only. While there is still no well-known, highly recognized professional association (with a large number of members) or well-known, comprehensive certification for data scientists as there is for actuaries (and I don’t think it is needed), there are more clear paths to reaching excellence in the profession, both as a company or as an employee. A physicist familiar with data could easily succeed with little on-the-job practice. There are companies open to hiring people from various backgrounds, which broadens the possibilities. And given the numerous poorly solved problems (they pop up faster than they can properly be solved), the future looks bright. Examples include counting the actual number of people once infected by Covid (requiring imputation methods) which might be twice as high as official numbers, assessing the efficiency of various Covid vaccines versus natural immunization, better detection of fake reviews / recommendations or fake news, or optimizing driving directions from Google map by including more criteria in the algorithm and taking into account HOV lanes, air quality, rarity of gas stations, and peak commute times (more on this in my next article about my 3,000 miles road trip using Google navigation). 

Renaissance Technologies is a good example: they have been working on quantitative trading since 1982, developing black-box strategies for high frequency trading, and mastering trading cost optimization. Many times, they had no idea and did not care why their automated self-learning trading system made some obscure trades (leveraging volatile patterns undetectable by humans or unused by competitors), yet it is by far the most successful hedge fund of all times, returning more than 66 percent annualized return (that is, per year, each year on average) for about 30 years. Yet they never hired traditional quants or data scientists, though some of their top executives came from IBM, with a background in computational linguistics. Many core employees had backgrounds in astronomy, physics, dynamical systems, and even pure number theory, but not in finance.

Incidentally, I have used many machine learning techniques and computational data science, processing huge volumes of multivariate data (numbers like integers or real numbers) with efficient algorithms, to try to pierce some of the deepest secrets in number theory. So I can easily imagine that a math background, especially one with strong experimental / probabilistic / computational number theory,  where you routinely uncover and leverage hard-to-find patterns in an ocean of seemingly very noisy data behaving worse than many messy business data sets (indeed dealing with chaotic processes), would be helpful in quantitative finance, and certainly elsewhere like fraud detection or risk management. I came to call these chaotic environments as gentle or controlled chaos, because in the end, they are less chaotic than they appear to be at first glance. I am sure many people in the business world can relate to that.

Conclusion

The job title data scientist might not be a great title, as it means so many things to different people. Better job titles include data science engineer, algorithmic data scientist, mathematical data scientists, computational data scientist, business data scientist, or analyst, reflecting the various fields that data science covers. There are still many unsolved problems, the list growing faster than that of solved problems, so the future looks bright. Some such as spam detection, maybe even automated translation, have seen considerable progress. Employers and employees have become better at matching with each other, and pay scale may not increase much more. Some tasks may disappear in the future, such as data cleaning, replaced by robots. Even coding might be absent in some jobs, or partially automated. For instance, the Data Science Central article that you read now was created on a platform in 2008 (by me, actually) without a single line of code. This will open more possibilities, as it frees a lot of time for the data scientist, to focus on higher level tasks.

To receive a weekly digest of our new articles, subscribe to our newsletter, here.

About the author:  Vincent Granville is a data science pioneer, mathematician, book author (Wiley), patent owner, former post-doc at Cambridge University, former VC-funded executive, with 20+ years of corporate experience including CNET, NBC, Visa, Wells Fargo, Microsoft, eBay. Vincent is also self-publisher at DataShaping.com, and founded and co-founded a few start-ups, including one with a successful exit (Data Science Central acquired by Tech Target). You can access Vincent’s articles and books, here. A selection of the most recent ones can be found on vgranville.com. 

Source Prolead brokers usa

Deep learning in biology and medicine

In this post, we examine applications of deep learning to three key biomedical problems:   patient classification, fundamental biological processes, and treatment of patients. The objective is to predict whether deep learning will transform these tasks.

The post is based on the very comprehensive paper “Opportunities and obstacles for deep learning in biology and medicine”.

The paper places a high bar i.e. on the lines of Andy Grove’s inflection point to refer to a change in technologies or environment that requires a business to be fundamentally reshaped.

The three classes of applications are described as follows:

Disease and patient categorization: the accurate classification of diseases and disease subtypes. In oncology, current “gold standard” approaches include histology, which requires interpretation by experts, or assessment of molecular markers such as cell surface receptors or gene expression.

Fundamental biological study: application of deep learning to fundamental biological questions using methods based on leveraging large amounts.  

Treatment of patients: new methods to recommend patient treatments, predict treatment outcomes, and guide the development of new therapies.

Within these, areas where deep learning plays a part for biology and medicine are

Deep learning and patient categorization            

• Imaging applications in healthcare
• Electronic health records
• Challenges and opportunities in patient categorization

Deep learning to study the fundamental biological processes underlying human disease

• Gene expression
• Splicing
• Transcription factors and RNA-binding proteins
• Promoters, enhancers, and related epigenomic tasks
• Micro-RNA binding
• Protein secondary and tertiary structure
• Morphological phenotypes
• Single-cell data
• Metagenomics
• Sequencing and variant calling

The impact of deep learning in treating disease and developing new treatments

• Clinical decision making
• Drug repositioning
• Drug development

There are a number of areas that impact deep learning in biology and medicine

These include:

• Evaluation
• Evaluation metrics for imbalanced classification
• Formulation of classification labels
• Formulation of a performance upper bound
• Interpretation and explainable results
• Hardware limitations and scaling
• Data, code, and model sharing
• Multimodal, multi-task, and transfer learning

I found two particularly interesting aspects: interpretability and data limitations. As per the paper:  

• deep learning lags behind most Bayesian models in terms of interpretability but the interpretability of deep learning is comparable to other widely-used machine learning methods such as random forests or SVMs.
• A lack of large-scale, high-quality, correctly labeled training data has impacted deep learning in nearly all applications discussed, from healthcare to genomics to drug discovery.
• The challenges of training complex, high- parameter neural networks from few examples are obvious, but uncertainty in the labels of those examples can be just as problematic.
• For some types of data, especially images, it is straightforward to augment training datasets by splitting a single labeled example into multiple
• Simulated or semi-synthetic training data has been employed in multiple biomedical domains, though many of these ideas are not specific to deep
• Data can be simulated to create negative examples when only positive training instances are available.
• Multimodal, multi-task, and transfer learning, can also combat data limitations to some

The authors conclude that deep learning has yet to revolutionize or definitively resolve any of these problems, but that even when improvement over a previous baseline has been modest, there are signs that deep learning methods may speed or aid human investigation.

This is a comprehensive paper which I recommend

“Opportunities and obstacles for deep learning in biology and medicine”.

The paper also has a github repository

Image source pixabay

Source Prolead brokers usa

4 Ways of Monetization Your Data

Ugh.  “Data Monetization” … a term that seems to confuse so many folks (probably thanks to me).  When most folks hear the phrase “data monetization”, they immediately think of “selling” their data.  And while there are some situations in which some organizations can successfully sell their data, there are actually more powerful, more common, and less risky ways for ANY organization to monetize – or derive business / economics value – from their data.

I’ve thought of 4 ways that organizations could monetize their data, and there are probably more.  Let’s review them.

There are organizations whose business model is based on selling third-party data[1].  Nielsen, Acxiom, Experian, Equifax and CoreLogic are companies whose business is the acquisition, aggregation, packaging, marketing, and selling of third-party data.  For example, Figure 1 shows the personal data that one can buy from Acxiom.

Figure 1:  Source: “Here are the data brokers quietly buying and selling your personal …”

Selling data requires dedicated technical and business organizations to acquire, cleanse, align, package, market, sell, support, and manage the third-party data for external consumption.  And there is a myriad of growing legal, privacy and ethical concerns to navigate, so a sizable legal team is also advised.

Some organizations can monetize their data by creating data services that facilitate the exchange of their data for something of value from other organizations. Walmart’s Retail Link® is an example of this sort of “data monetization through exchange.”

Walmart’s Retail Link® exchanges (for a price) Walmart’s point-of-sales (POS) data with its Consumer Packaged Goods (CPG) manufacturing partners such as Procter & Gamble, PepsiCo, and Unilever. Retail Link provides the CPG manufacturers access to that manufacturer’s specific product sell-through data by SKU, by hour, by store as well as inventory on-hand, gross margin achieved, inventory turns, in-stock percentages, and Gross Margin Return on Inventory Investment (Figure 2).

Figure 2: Sample report courtesy of  Trend Results[2]

Unfortunately, not all organizations have the clout and financial and technology resources of a Walmart to dictate this sort of relationship.  Plus, Walmart invests a significant amount of time, money, and people resources to develop, support, and upgrade Retail Link.  In that aspect, Walmart looks and behaves like an enterprise software vendor.

But for organizations that lack the clout, finances, and technology expertise of a Walmart, there are other more profitable, less risky “monetization” options.

Probably the most common way for organizations to monetize or derive value from their data is in the application of their data to optimize the organization’s most important business and operational use cases.  And the funny thing here is that it isn’t really the data that one uses to monetize an organization’s internal use cases, it’s actually the customer, product, and operational insights that is used to optimize these use cases.

Insights Monetization is about leveraging the customer, product, and operational insights (predicted behavioral and performance propensities) buried in your data sources to optimize and/or reengineer key business and operational processes, mitigate (compliance, regulatory, and business) risks, create new revenue opportunities (such new products, services, audiences, channels, markets, partnerships, consumption models, etc.), and construct a more compelling, differentiated customer experience (Figure 3).

Figure 3: Data Monetization through Internal Use Case Optimization

To apply “Insights” to drive internal Use Case Optimization requires some key concepts:

(1) Nanoeconomics.  Nanoeconomics is the economics of individualized human and/or device predicted behavioral or performance propensities.  Nanoeconomics helps organizations transition from overly generalized decisions based upon averages to precision decisions based upon the predicted propensities, patterns, and trends of individual humans or devices.

(2) Analytic Profiles provide an asset model for capturing and codifying the organization’s customer, product, and operational analytic insights in a way that facilities the sharing and refinement of those analytic insights across multiple use cases. An Analytic Profile captures metrics, predictive indicators, segments, analytic scores, and business rules that codify the behaviors, preferences, propensities, inclinations, tendencies, interests, associations, and affiliations for the organization’s key business entities such as customers, patients, students, athletes, jet engines, cars, locomotives, CAT scanners, and wind turbines (Figure 4).

Figure 4: Analytic Profiles

(3) Use Cases are comprised of Decisions clustered around a common Key Performance Indicator (KPI) where Decisions are a conclusion or resolution reached after analysis that leads to an informed action.  Sample use cases include reduce customer attrition, improve operational uptime, and optimize asset utilization. Analytic Profiles are used to optimize the organization’s top priority use cases.

See the blog “Costs of Being an Analytics Laggard…And Path to Becoming a Leader” for more details on the integration of Nanoeconomics, Analytic Profiles, and Use Cases to drive the organization’s internal Insights Monetization efforts.

Finally, some organizations are fortunate to have a broad overview of their market.  They know what products or services are hot, which ones are in decline, and who is buying and not buying those products or services, and what sorts of marketing and actions works best for driving engagement.  For those organizations, there is a fourth way to monetize their data – by packaging and selling “decisions” in the form of Data Products to their customers, partners, and suppliers (Figure 4).

Figure 5: Data Monetization thru Selling “Decisions” via Data Products

Instead of just selling or exchanging data with your partners and suppliers, these organizations leverage their broader market perspective to build data products that help their customers, partners, and suppliers optimize their key business and operational decisions in areas such as:

• Demand Planning
• Inventory Management
• Logistics Optimization
• Campaign Marketing
• New Product Introductions
• Customer Retention

To sell Data Products requires an intimate understanding of your partners and suppliers’ business models and the key decisions that they trying to make.

For example, a large digital media company has enough customer, product, and operational insights across its ad network to help their customers and business partners (ad agencies) make better decisions in the areas of ad placement, dayparting, audience targeting and retargeting, and keyword bidding.  The digital media company could build a data product that delivers operational recommendations that optimize their customers’ and business partners’ digital marketing spend (Figure 6).

Figure 6: Packaging and Selling “Decisions”

Any organization that has a broad view of a market (think OpenTable and GrubHub for restaurants, Fandango for movies, Travelocity or Orbitz for travel and entertainment) could build such a data product for their customers, industry partners, and suppliers.

Don’t miss the boat on Data Monetization.  Focusing just on trying to sell your data is not practical for the vast majority of companies whose business model is not in the acquisition, aggregation, selling, and supporting of third-party data sources. And creating “data exchanges” really only works if your organization has enough industry market share and clout to dictate the terms and conditions of these sorts of relationships.

However, any organization can monetize their customer, product, and operational insights.  The easiest is in the application of these insights to optimize the organization’s internal use cases.  And organizations can go one step further and build data products that package these insights into recommendations that support their partners’ and suppliers’ most important business decisions (Figure 7).

Figure 7: 4 Types of Data Monetization

Best of luck on your data monetization journey!

[1] Third-party data is any data that is collected by an entity that does not have a direct relationship with the user whose data is being collected

[2] Trend Results is in no way associated with or endorsed by Wal-Mart Stores, Inc. All references to Wal-Mart Stores, Inc. trademarks and brands are used in strict accordance with the Fair Use Doctrine and are not intended to imply any affiliation of Trend Results with Wal-Mart Stores, Inc. Retail Link is a registered trademark of Wal-Mart Stores, Inc.

Source Prolead brokers usa

Advantages of Using Trading Robots on Quantitative Hedge Funds

“The time will come when no human investment manager will be able to beat the computer,” David Siegel (co-founder Two Sigma)

Generally, hedge funds engage in investing for the long term rather than day trading. In that context, the investment requires developing a macro or microeconomic thesis, understanding the market, and utilizing this perception to employ a vision, building a position, and then comes the part of holding and managing that position for a while, for a few days to often months.

A quantitative analyst explores hundreds of different pieces of information to predict an eligible output to recognize and measure attractive long-term or short-term positions in the market. In most cases, an algorithm can process more information than human analysts and keep track of that information in its database. 

Likewise, there are some significant advantages the trading robots offer for quantitative hedge funds. They are:

Unbiased Trading Ability Based on Symmetrical Analysis:

Hedge funds fundamentally analyze and utilize the economic and financial data to evaluate the possible attempts. In the process, the entire investment method is a research-driven process that is completely based on the symmetric orientation of strategies. 

Trading robots are insensitive, impartial about information, and perform all the trades based on substantial and symmetrical data analysis. Therefore, nothing and nothing can halt trading robots from sticking to the algorithms.

The most challenging task for a regular trader is to stay on the course of discipline and maintain the strategy without getting distracted. However, a trading robot successfully remains on the trail of discipline and strategy. 

Here the hedge fund manager plays an indispensable role because the robots may burn the fund if the market responds differently than the program. In such cases, the manager makes the call and trades manually or upgrades the instructions to the robots considering the situation. Therefore, an experienced hedge fund manager with the assistance of effective trading robots can achieve a significant return.

Backtesting Capacity:

One of the most significant advantages of automated trading for hedge funds is Backtesting. It’s a method of making the robot perform on historical data. Thus, you can have the performance graph of the robot in multiple previous and various market scenarios from different times. 

In general, while Backtesting, the hedge fund managers operate the trading robot through the previous market Uptrends, Downtrends, or other sideway trends. Consequently, the outcome confers the strength of managing trades in comparable environments.

Availability of Efficient Automated Trading Solutions 

Nowadays, many trading platforms offer complete automated solutions for hedge funds. But among them, the MetaTrader platform provides the most sophisticated and efficient automated trading capabilities to operate large amounts of funds tirelessly. And the accretion of similar automated solutions making automated hedge fund management more accessible than ever. 

The MQL Market provides a collection of more than 13000 pre-made trading robots and solutions for legitimate automated trading! Therefore if you are looking for a trading-related solution, trading robot, or anything related to automated trading, the MQL community is there to assist you. 

If you are using the MetaTrader 5 for hedge funds, you are about to get the advantages of choosing from thousands of different trading robots for a very efficient trading solution.

Characteristics of The Most Profitable Trading Robot

Numerous trading robots have helped users earn a lot of money through automated trading. Also, some robots promise profit and a proficient gain in the market, although they end up burning out all the cash. Likewise, there are thousands of similar disappointments where the trader lost the investment depending on either fraudulent or unprofitable robots. 

Running a trading robot can be tricky as its algorithm determines the most reliable results from simultaneous events, and concurrently all trading robots claim to be better than the others. Therefore, the traders usually use specific indicators to find the most profitable trading robot for them. 

There are no such standards set yet to determine the most profitable robots. However, several functionalities determine the performance of a trading robot. Therefore, at this point, we are about to discuss a few characteristics an ideal trading robot should contain:

Efficient Money Management: As a trading robot remains active 24/7 and the trader not being available always creates some space for the robot to manage money. One of the most vital responsibilities of a trading robot is to take profits and stop losses. Therefore, maintaining the take-in while gaining profit and saving the account balance performing hard stop loss while losing money becomes a full-time duty for the automated system. 

AI & Machine Learning: An expert robot should be capable of identifying versatile trading market situations using Artificial intelligence. As a result, the robot would apply different patterns in trading to adapt to the sudden changes in the trading market. Whether it’s the reality or the trading market, revolution is the key to survival. 

Sustainable Profit: A proficient trading robot will provide you a regular steady flow of profit rather than forming an asymmetrical growth. Usually, a prominent trading robot should generate an 8-12% monthly gain. Somewhat robots may gain 20% or 25%, but the more steady, the better it is. 

Fewer Drawdowns: When it comes to Drawdowns, anything less than 20% is excellent for an ideal trading robot. A robust applied strategy and risk management can keep the Drawdown level between 2% to 20%. 

Bug-free Lifespan: Some very efficient trading robots often start acting differently after six months to a year. That happens because of vulnerability to bugs. Therefore, a proficient trading robot should have strong protection against bugs. Otherwise, they could take you down or even become scammers!  

User friendly: A functional trading robot should be user-friendly considering the significant numbers of beginner automated traders. 

User Reviews: The better a trading robot, the better the users speak about it. In the MQL community, there are dozens of thousands pre-made trading robots available with plenty of user comments regarding their performance and efficiency. So choose the one that comes with more positive comments.

Source Prolead brokers usa