MapReduce with Python

MapReduce is a programming model that enables large volumes of data to be processed and generated by dividing work into independent tasks and executing the tasks in parallel across a cluster of machines. The MapReduce programming style was inspired by the functional programming constructs map and reduce, which are commonly used to process lists of […]

DH2i Achieves VMware Ready Status

DH2i Company, recently named a Cool Vendor by Gartner in the “Cool Vendors in Servers and Virtualization, 2015” report, today announced that its DxEnterprise container management software for Microsoft Windows Server has achieved VMware Ready status. This designation indicates that DxEnterprise has undergone detailed test procedures and is supported on VMware vSphere for production environments. […]

Working with Snakebite in Python

Snakebite is a Python package, created by Spotify, that provides a Python client library, allowing HDFS to be accessed programmatically from Python applications. The client library uses protobuf messages to communicate directly with the NameNode. The Snakebite package also includes a command-line interface for HDFS that is based on the client library. This section describes […]

HDFS Command Reference

The commands demonstrated in this section are the basic file operations needed to begin using HDFS. Below is a full listing of file manipulation commands possible with hdfs dfs. This listing can also be displayed from the command line by specifying hdfs dfs without any arguments. To get help with a specific option, use either […]

Creating a Directory in HDFS

Home directories within HDFS are stored in /user/$HOME. From the previous example with -ls, it can be seen that the /user directory does not currently exist. To create the /user directory within HDFS, use the -mkdir command: $ hdfs dfs -mkdir /user To make a home directory for the current user, hduser, use the -mkdir […]

Interacting with HDFS

Interacting with Hadoop Distributed File System (HDFS) is primarily performed from the command line using the script named hdfs. The hdfs script has the following usage: $ hdfs COMMAND [-option <arg>] The COMMAND argument instructs which functionality of HDFS will be used. The -option argument is the name of a specific option for the specified […]

Introducing Hadoop Distributed File System (HDFS)

The Hadoop Distributed File System (HDFS) is a Java-based distributed, scalable, and portable filesystem designed to span large clusters of commodity servers. The design of HDFS is based on GFS, the Google File System, which is described in a paper published by Google. Like many other distributed filesystems, HDFS holds a large amount of data […]

Debugging teams: Creating relationships to get things done

Debugging teams: Creating relationships to get things done

It’s the same story. We idolize him, but the fact is that he didn’t win every basketball game by himself. His true genius was in the way he worked with his team. The team’s coach, Phil Jackson, was extremely clever – his coaching techniques are legendary: he recognized that one player alone never wins a […]

Debugging teams: Creating relationships to get things done

Debugging teams: Creating relationships to get things done

Since this is a book about the social perils of creative development, it makes sense to focus on the one variable you definitely have control of: you. People are inherently imperfect. But before you can understand the bugs in your coworkers, you need to understand the bugs in yourself. We’re going to ask you to […]

Use of Smartphone in our Daily Life

What is the most important thing in your life? A thing which is always with you, which goes with you to work, goes with you on your business meetings, on trips, at home, and sometimes even in your sleep. It is your connection to the world at large. Perhaps, you have guessed it already? If […]