One of the challenges of large scale data analysis is being able to get the value from data with least effort. Doing that often involves multiple stages: provisioning infrastructure, accessing or moving data, transforming or filtering data, analyzing and learning from data, automating the data pipelines, connecting with other services that provide input or consume the output data, and more. There are quite a few tools available to solve these questions, but it’s usually difficult to have them all in one place and easily connected.

If this article was helpful or interesting to you, follow @lenadroid on Twitter.

Introduction

This is…

We are delighted to announce the Applied Cloud Stories initiative by Microsoft!

📚 What Is Applied Cloud Stories?

Do you work with open-source? Are you passionate about machine learning or data science? Do you have stories to share about solving scale or data challenges? Are you investing time and effort so that you and your teams can ship code better, faster, with more confidence? Do you work with Java, Python, JavaScript, Go, Rust, or other exciting languages in the cloud? Are you active in computer science research and love when theory meets practice?

We would love to hear your story.

Applied Cloud Stories is an open…

F# is an open-source, functional, general-purpose programming language that originates from Microsoft Research and is used by many engineers across the globe. F# is actively developed by the open-source community and Microsoft’s developer division.

One of the great examples showing F# strengths is Azure Storage Type Provider. It helps ensure strongly-typed access to Azure storage blobs, tables, and queues with automatic generation of the schema, and makes programming more efficient and less error-prone.

open FSharp.Azure.StorageTypeProvider

// Get a handle to my local storage emulator
type Azure = AzureTypeProvider<"UseDevelopmentStorage=true">

// Navigate through the containers to a specific file
// and read the contents.
let blobContents =…

For more useful information on technical topics follow @lenadroid on Twitter and subscribe to her YouTube channel.

For detailed documentation of Azure topics covered in this article, take a look at Azure Functions, Managed Identities documentation, and Azure CLI. To follow through examples, get started with Free Trial.

Introduction

In this article you will learn about using Managed Identities for secure and convenient approach to authenticate to Azure services from Serverless Functions without managing or storing any access keys or credentials!

We will create an Azure Function, obtain an access token from local service identity endpoint, and we will use the…

For detailed documentation of Azure topics covered in this article, take a look at Managed Identities documentation, Azure Linux Virtual Machines, and Azure CLI. To follow through examples, get started with Free Trial.

Introduction

In this article we are discussing Azure Managed Identities! Why is it a good idea to use them? How can they help make access to Azure resources more secure and less error-prone?

TL;DR: They can significantly improve authentication process between cloud resources, reduce operational overhead and vulnerability of managing and storing authentication credentials.

Note: Managed Identities in Azure are similar to IAM roles for EC2 instances on…

Apache Spark it is an open-source fault tolerant distributed computing framework and scalable data processing system, which is part of Apache Software Foundation. Spark is an actively maintained project with vibrant community that consists of multiple components with Spark Core as a foundation of it. Other components include Spark SQL, Spark Streaming, Spark Structured Streaming, MLlib Machine Learning Library and GraphX.

Note:

“Apache Spark is a powerful open source processing engine built around speed, ease of use, and sophisticated analytics. It was originally developed at UC Berkeley in 2009.” — Databricks

Use cases for Spark include data processing, analytics, and…

The year of 2017 has started with continuation of me working at Microsoft Research on a super fascinating project “Springfield”, which is now called “Microsoft Security Risk Detection”. As you might have guessed, I have been a functional programming enthusiast for quite a long time and was absorbed by functional programmer-ing in F# at Microsoft Research as a Senior Wizard Engineer, building an intelligent cloud service for optimized infrastructure provisioning, and dedicated significant amount of effort to enhancing machine storage strategy.

I have had a privilege of meeting and talking to Leslie Lamport during his workshop in February at AI+R…

In this article, I am going to cover several topics, related to distributed databases, containers, cloud and functional programming.

Prerequisites

I already have a Kubernetes cluster provisioned in Azure. Look here if you would like to create one.

Distributed databases — systems that require stable persistence

If we think about it, a database and data are most valuable parts of any system. Nobody wants to deal with data leaks, instability and data loss, especially. Nobody wants to spend days and nights recovering from an operational or architectural mistake that could cost significant amounts of money. …