What is data that is organized into categories in which there is no order?

In computer science, a data structure is a particular way of organising and storing data in a computer such that it can be accessed and modified efficiently. More precisely, a data structure is a collection of data values, the relationships among them, and the functions or operations that can be applied to the data.

Three different data structures

For the analysis of data, it is important to understand that there are three common types of data structures:

Structured Data

Structured data is data that adheres to a pre-defined data model and is therefore straightforward to analyse. Structured data conforms to a tabular format with relationship between the different rows and columns. Common examples of structured data are Excel files or SQL databases. Each of these have structured rows and columns that can be sorted.

Structured data depends on the existence of a data model – a model of how data can be stored, processed and accessed. Because of a data model, each field is discrete and can be accesses separately or jointly along with data from other fields. This makes structured data extremely powerful: it is possible to quickly aggregate data from various locations in the database.

Structured data is is considered the most ‘traditional’ form of data storage, since the earliest versions of database management systems (DBMS) were able to store, process and access structured data.

Unstructured Data

Unstructured data is information that either does not have a predefined data model or is not organised in a pre-defined manner. Unstructured information is typically text-heavy, but may contain data such as dates, numbers, and facts as well. This results in irregularities and ambiguities that make it difficult to understand using traditional programs as compared to data stored in structured databases. Common examples of unstructured data include audio, video files or No-SQL databases.

The ability to store and process unstructured data has greatly grown in recent years, with many new technologies and tools coming to the market that are able to store specialised types of unstructured data. MongoDB, for example, is optimised to store documents. Apache Giraph, as an opposite example, is optimised for storing relationships between nodes.

The ability to analyse unstructured data is especially relevant in the context of Big Data, since a large part of data in organisations is unstructured. Think about pictures, videos or PDF documents. The ability to extract value from unstructured data is one of main drivers behind the quick growth of Big Data.

Semi-structured Data

Semi-structured data is a form of structured data that does not conform with the formal structure of data models associated with relational databases or other forms of data tables, but nonetheless contain tags or other markers to separate semantic elements and enforce hierarchies of records and fields within the data. Therefore, it is also known as self-describing structure. Examples of semi-structured data include JSON and XML are forms of semi-structured data.

The reason that this third category exists (between structured and unstructured data) is because semi-structured data is considerably easier to analyse than unstructured data. Many Big Data solutions and tools have the ability to ‘read’ and process either JSON or XML. This reduces the complexity to analyse structured data, compared to unstructured data.

Metadata – Data about Data

A last category of data type is metadata. From a technical point of view, this is not a separate data structure, but it is one of the most important elements for Big Data analysis and big data solutions. Metadata is data about data. It provides additional information about a specific set of data.

In a set of photographs, for example, metadata could describe when and where the photos were taken. The metadata then provides fields for dates and locations which, by themselves, can be considered structured data. Because of this reason, metadata is frequently used by Big Data solutions for initial analysis.

All data is not created equal. Some data is structured, but most of it is unstructured. Structured and unstructured data is sourced, collected and scaled in different ways, and each one resides in a different type of database.

In this article, we’ll take a deep dive into both types so that you can get the most out of your data.

What is structured data?

Structured data — typically categorized as quantitative data — is highly organized and easily decipherable by machine learning algorithms. Developed by IBM in 1974, structured query language (SQL) is the programming language used to manage structured data. By using a relational (SQL) database, business users can quickly input, search and manipulate structured data.

Pros and cons of structured data

Examples of structured data include dates, names, addresses, credit card numbers, etc. Their benefits are tied to ease of use and access, while liabilities revolve around data inflexibility:

Pros

• Easily used by machine learning (ML) algorithms: The specific and organized architecture of structured data eases manipulation and querying of ML data.
• Easily used by business users: Structured data does not require an in-depth understanding of different types of data and how they function. With a basic understanding of the topic relative to the data, users can easily access and interpret the data.
• Accessible by more tools: Since structured data predates unstructured data, there are more tools available for using and analyzing structured data.

Cons

• Limited usage: Data with a predefined structure can only be used for its intended purpose, which limits its flexibility and usability.
• Limited storage options: Structured data is generally stored in data storage systems with rigid schemas (e.g., “data warehouses”). Therefore, changes in data requirements necessitate an update of all structured data, which leads to a massive expenditure of time and resources.

Structured data tools

• OLAP: Performs high-speed, multidimensional data analysis from unified, centralized data stores.
• SQLite: Implements a self-contained, serverless, zero-configuration, transactional relational database engine.
• MySQL: Embeds data into mass-deployed software, particularly mission-critical, heavy-load production system.
• PostgreSQL: Supports SQL and JSON querying as well as high-tier programming languages (C/C+, Java, Python, etc.).

Use cases for structured data

• Customer relationship management (CRM): CRM software runs structured data through analytical tools to create datasets that reveal customer behavior patterns and trends.
• Online booking: Hotel and ticket reservation data (e.g., dates, prices, destinations, etc.) fits the “rows and columns” format indicative of the pre-defined data model.
• Accounting: Accounting firms or departments use structured data to process and record financial transactions.

What is unstructured data?

Unstructured data, typically categorized as qualitative data, cannot be processed and analyzed via conventional data tools and methods. Since unstructured data does not have a predefined data model, it is best managed in non-relational (NoSQL) databases. Another way to manage unstructured data is to use data lakes to preserve it in raw form.

The importance of unstructured data is rapidly increasing. Recent projections indicate that unstructured data is over 80% of all enterprise data, while 95% of businesses prioritize unstructured data management.

Pros and cons of unstructured data

Examples of unstructured data include text, mobile activity, social media posts, Internet of Things (IoT) sensor data, etc. Their benefits involve advantages in format, speed and storage, while liabilities revolve around expertise and available resources:

Pros

• Native format: Unstructured data, stored in its native format, remains undefined until needed. Its adaptability increases file formats in the database, which widens the data pool and enables data scientists to prepare and analyze only the data they need.
• Fast accumulation rates: Since there is no need to predefine the data, it can be collected quickly and easily.
• Data lake storage: Allows for massive storage and pay-as-you-use pricing, which cuts costs and eases scalability.

Cons

• Requires expertise: Due to its undefined/non-formatted nature, data science expertise is required to prepare and analyze unstructured data. This is beneficial to data analysts but alienates unspecialized business users who may not fully understand specialized data topics or how to utilize their data.
• Specialized tools: Specialized tools are required to manipulate unstructured data, which limits product choices for data managers.

Unstructured data tools

• MongoDB: Uses flexible documents to process data for cross-platform applications and services.
• DynamoDB: Delivers single-digit millisecond performance at any scale via built-in security, in-memory caching and backup and restore.
• Hadoop: Provides distributed processing of large data sets using simple programming models and no formatting requirements.
• Azure: Enables agile cloud computing for creating and managing apps through Microsoft’s data centers.

Use cases for unstructured data

• Data mining: Enables businesses to use unstructured data to identify consumer behavior, product sentiment, and purchasing patterns to better accommodate their customer base.
• Predictive data analytics: Alert businesses of important activity ahead of time so they can properly plan and accordingly adjust to significant market shifts.
• Chatbots: Perform text analysis to route customer questions to the appropriate answer sources.

What are the key differences between structured and unstructured data?

While structured (quantitative) data gives a “birds-eye view” of customers, unstructured (qualitative) data provides a deeper understanding of customer behavior and intent. Let’s explore some of the key areas of difference and their implications:

• Sources: Structured data is sourced from GPS sensors, online forms, network logs, web server logs, OLTP systems, etc., whereas unstructured data sources include email messages, word-processing documents, PDF files, etc.
• Forms: Structured data consists of numbers and values, whereas unstructured data consists of sensors, text files, audio and video files, etc.
• Models: Structured data has a predefined data model and is formatted to a set data structure before being placed in data storage (e.g., schema-on-write), whereas unstructured data is stored in its native format and not processed until it is used (e.g., schema-on-read).
• Storage: Structured data is stored in tabular formats (e.g., excel sheets or SQL databases) that require less storage space. It can be stored in data warehouses, which makes it highly scalable. Unstructured data, on the other hand, is stored as media files or NoSQL databases, which require more space. It can be stored in data lakes which makes it difficult to scale.
• Uses: Structured data is used in machine learning (ML) and drives its algorithms, whereas unstructured data is used in natural language processing (NLP) and text mining.

What is semi-structured data?

Semi-structured data (e.g., JSON, CSV, XML) is the “bridge” between structured and unstructured data. It does not have a predefined data model and is more complex than structured data, yet easier to store than unstructured data.

Semi-structured data uses “metadata” (e.g., tags and semantic markers) to identify specific data characteristics and scale data into records and preset fields. Metadata ultimately enables semi-structured data to be better cataloged, searched and analyzed than unstructured data.

• Example of metadata usage: An online article displays a headline, a snippet, a featured image, image alt-text, slug, etc., which helps differentiate one piece of web content from similar pieces.
• Example of semi-structured data vs. structured data: A tab-delimited file containing customer data versus a database containing CRM tables.
• Example of semi-structured data vs. unstructured data: A tab-delimited file versus a list of comments from a customer’s Instagram.

The future of data

Recent developments in artificial intelligence (AI) and machine learning (ML) are driving the future wave of data, which is enhancing business intelligence and advancing industrial innovation. In particular, the data formats and models covered in this article are helping business users to do the following:

• Analyze digital communications for compliance: Pattern recognition and email threading analysis software that can search email and chat data for potential noncompliance.
• Track high-volume customer conversations in social media: Text analytics and sentiment analysis that enables monitoring of marketing campaign results and identifying online threats.
• Gain new marketing intelligence: ML analytics tools that can quickly cover massive amounts of data to help businesses analyze customer behavior.

Furthermore, smart and efficient usage of data formats and models can help you with the following:

• Understand customer needs at a deeper level to better serve them
• Create more focused and targeted marketing campaigns
• Track current metrics and create new ones
• Create better product opportunities and offerings
• Reduce operational costs

Structured and unstructured data and IBM

Whether you are a seasoned data expert or a novice business owner, being able to handle all forms of data is conducive to your success. By leveraging structured, semi-structured and unstructured data options, you can perform optimal data management that will ultimately benefit your mission.

To better understand data storage options for whatever kind of data best serves you, check out IBM Cloud Databases.

What are categories of data called?

Which type of data are categories with an order?

What is ordinal and nominal data?

What are the 4 categories of data?