Module 5:
Consumer Search and Resource Discovery, information search and Retrieval, Electronic commerce
catalogs or directories, Information Filtering. Multimedia and Digital video, Key Multimedia concepts,
Digital Video & Electronic Commerce, Desktop Video Processing, Desktop Video Conferencing.
Consumer Search and Resource Discovery
INFORMATION SEARCH AND RETRIEVAL:
Information retrieval (IR) is a process that facilitates the effective and efficient retrieval
of relevant information from large collections of unstructured or semi-structured
data. IR systems assist in searching for, locating, and presenting information that matches a
user's search query or information need.
As the dominant form of information access, information retrieval is relied upon by billions
of people every day who use search engines. Deploying various models, algorithms, and
increasingly advanced techniques (think: vector search), information retrieval systems
enable search access to a vast and growing array of sources, including documents, items
within documents, metadata, and databases of texts, images, videos, and sounds.
A brief history of information retrieval
The roots of information retrieval can be traced back to ancient times, when libraries and
archives were established to organize and store information, including the indexing and
alphabetization of scholarly work. By the 1800s, punch cards were being used to process
information, and in 1931, Emanuel Goldberg received a patent for the first successful
electromechanical document retrieval device, known as the "Statistical Machine," designed
for searching through data encoded on film.
Information retrieval began to formalize into what would become a scientific discipline in the
mid-20th century, alongside the development of modern computers. Gerard Salton and Hans
Peter Luhn pioneered early models for automated document retrieval. Salton and colleagues
at Cornell created the SMART Information Retrieval System in the 1960s, a milestone in the
field credited with laying the foundation for modern IR techniques and key concepts,
including the term-document matrix, the vector space model, relevance feedback, and
Rocchio classification.
By the 1970s, with the emergence of more advanced retrieval techniques, probabilistic
models, and fully articulated vector processing frameworks, the field had advanced
significantly. With the advent of search engines in the late 1990s, IR systems and models that
were once mostly the province of academia, institutions, and libraries were put into wide
service.
Types of information retrieval models
Different types of information retrieval models are designed to tackle specific challenges and
establish processes to retrieve relevant information. There are classical models that form the
foundations of the field, non-classical models that attempt to address the limitations of
traditional approaches, and alternative IR models that go even further, often by integrating
advanced technologies like machine learning and language models. On a general level, the
most common types of information retrieval models include:
Boolean model:One of the simplest and earliest information retrieval models, the Boolean
model is based on Boolean logic, which uses operators including AND, OR, and NOT to
combine query terms. Documents are represented as sets of terms, and a query is processed to
identify documents that match the specified conditions. Though it’s effective for precise
query matches, the Boolean model can't rank documents based on relevance or deliver partial
matches.
Vector space model:In this model, documents, and queries are represented as vectors in a
multi-dimensional space. Each dimension corresponds to a unique term, and the value in each
dimension represents the importance and frequency of the term in the document or query.
The cosine similarity between the query vector and document vectors is calculated to
determine the relevance of documents to the query. Developed in part to address the
disadvantages of the Boolean model, the vector space model can provide ranked results based
on relevance scores and is widely used in text retrieval.
Probabilistic model:This model estimates the probability that a document is relevant to a
given query. It considers factors like term frequency and document length to calculate
relevance probabilities. It is particularly useful for dealing with large amounts of data.
Because it works with weighted statistics, the model is an ideal fit for providing ranked
results.
Latent semantic indexing (LSI):LSI uses singular value decomposition (SVD) to capture
the semantic relationships between terms and documents. Like semantic search, semantic
indexing uses intent and context to identify conceptually related documents even if they don't
share exact terms. This key capability makes LSI useful for extracting the contextual meaning
of words in a body of text.
Okapi BM25:One of the more popular variants of the probabilistic model, BM25 is a search
relevance ranking function. It is used by search engines to estimate the relevance of a
document to a search query. It ranks a set of documents based on the query terms appearing
in each document, regardless of the inter-relationship between the terms within a document,
and consists of many scoring functions with different components and parameters. The BM
stands for "best matching."
Why is information retrieval important?
In the Information Age, data is generated every single second at a scale that was once
unimaginable. Without a viable means of accessing information, the data is effectively
useless. IR systems ensure that users can obtain the relevant information they need amidst the
growing noise of information overload.
Information retrieval plays a vital role across nearly every industry and domain in the modern
world, from academia and ecommerce to healthcare and defense. It's a human-machine
interface that aids in decision-making, research, and knowledge discovery, on both an
enterprise and personal level. From searching our localized desktops to discovering the news
of the world, or from genomic research to spam filtering, information retrieval is fundamental
to nearly every facet of our lives.
Search engines rely on information retrieval models to deliver accurate search results.
Ecommerce platforms use retrieval models to recommend products based on user preferences
and behavior. Digital libraries rely on information retrieval science to help users do research.
In healthcare, IR systems assist in searching databases for relevant patient records, medical
research, and treatment protocols. And legal professionals use information retrieval to comb
through large volumes of legal cases for precedents.
How does an information retrieval system work?
The information retrieval process is typically triggered when a user enters a formal query into
a system stating their information needs. The IR system creates an index of documents in a
content collection or information database. Data objects, including those from text
documents, images, audio, and videos, are processed to extract relevant terms and surrogate
data, and data structures are used to efficiently store and retrieve those entities.
When a user submits a query, the system processes it to identify relevant terms and determine
their importance. The system then ranks documents based on their relevance to the query. In
many cases, IR models and algorithms are used to calculate a numeric score based on how
well each object in the collection or database matches the query. Many queries won’t be an
exact match: the most relevant documents are presented to the user in a ranked list. These
ranked results represent one of the key differences between information retrieval search and
database search.
Main components of an information retrieval system
An information retrieval system consists of several key components:
Document collection:The set of documents that the system can retrieve information from.
Indexing component:Source data and documents are processed to create an index, mapping
terms and data to the documents that contain them — often in a dedicated, optimized data
structure.
Query processor:The query processor analyzes user queries and keywords and prepares
them for matching against the indexed entities.
Ranking algorithm:The ranking algorithm determines the relevance of documents to a query
and assigns them scores. The most common is the BM25 (Best Match 25) ranking algorithm,
which is notable for its modified approach to term frequency that avoids oversaturating the
document with keywords and repeated terms.
User interface:The UI is the display through which users interact with the system, submit
queries, and are presented with results. Here, results can be tuned based on how well they
serve a user query. In some cases, mechanisms might allow users to provide feedback on the
relevance of retrieved documents, which can be used to improve future retrievals.
Benefits of information retrieval
Significant benefits of information retrieval models include:
Efficient information access: Above all, IR systems save people untold amounts of time
and effort. Information retrieval enables users to quickly access relevant information
without manually searching through vast troves of documents and data.
Knowledge discovery: Information retrieval is a powerful tool that allows us to make
sense of data. With IR, users can identify trends, patterns, and relationships within data
that might not be initially apparent.
Personalization: Some IR systems can tailor results in a meaningful way to individual
users based on their preferences and behaviors.
Decision support: Professionals are empowered to make informed decisions with access
to the most pertinent information when they need it.
Challenges and limitations of information retrieval
Despite significant advancements, information retrieval has never been perfect. Known
issues, challenges, and limitations remain, including:
Ambiguity Natural language is inherently ambiguous, making it challenging to accurately
interpret user queries. Similar issues of vagueness and uncertainty can affect the indexing and
evaluation process, particularly with objects like images and videos.
Relevance Determining relevance is subjective and can vary based on user context and
intent. Criteria used to determine worth and significance may be governed by a set of
imperfect, general standards that don’t reflect the specific needs of the individual user.
Semantic gaps Retrieval systems can struggle to capture the deeper meaning of content due
to the gap between textual representation and human understanding. Lack of clarity in
information and user expression represents a major obstacle to successful IR.
Advanced natural language processing powered by AI seeks to close those semantic and
ambiguity gaps.
Scalability As data volumes increase, maintaining efficient and effective retrieval and
indexing becomes more complex, with more and more resources and computing power
required.
Future trends in information retrieval
With recent breakthroughs in generative AI and machine learning, information retrieval as
we know it may be on the verge of transformational change.
Advanced machine learning techniques are already enhancing retrieval by learning from
user interactions and adapting to changing contexts, locations, and preferences. Improved
natural language processing and semantic analysis create a better understanding of user
queries and document content. Retrieval systems are also evolving to more effectively handle
the ever-growing deluge of multimedia content.
The impact of generative AI on information retrieval has the potential to be revolutionary.
Instead of the ranked list of results we’re used to, which requires manually sorting through
existing links and documents to find what we’re looking for, we’ll receive actual answers to
our questions. The context will be carried over from question to question, allowing for
complex, conversational, multi-step inquiries, with the barriers of human language processing
and intent all but erased. Rather than piecing together answers ourselves, search engines will
do the work for us, synthesizing information into specific, customized results in the form of
original content that provides exactly what we need — and nothing we don't.
Electronic commerce catalogs or directories:
It’s no secret that people nowadays prefer to buy almost everything online. In 2018, as per
Internet Retailer’s analysis, online business accounted for over 50% of global web sales. It
means more companies are bringing their assets online, and more people are purchasing from
these marketplaces. So, whether you are selling online or offline, your business needs
attractive ecommerce catalogs to showcase your products and services.
It is one of the efficient ways to grab the viewer’s eye and increase sales chances. One of the
most significant ingredients for your ecommerce product catalog is the professional product
images that help you attract the viewer’s attention.
Businesses are now inclined towards creating elegant, personalized, and intuitively designed
digital product catalogs to help users to search more precisely and effectively. This is
primarily fueled by the massive growth in online traffic on eCommerce websites and online
marketplaces, the majority being mobile traffic.
An electronic catalog in ecommerce is one of the principal components of any business that
reinforces brand and product awareness. It consolidates a database of information on all the
available products to be sold or launched. It helps you inform and educate your audience
about the product and services and grab the viewer’s attention towards the product.
While studying modern consumer behavior, Big Commerce revealed that 82 percent of cell
phone users research purchases online before buying the product, even when they anticipate
buying it from a store.
As per Accenture, 72 percent of B2B organizations prefer to conduct product research online
before buying the products. It determines how necessary e-commerce merchandising has
become for businesses if you want to foster, sell your product, or bring it online decisively.
What is an Ecommerce Catalog?
An eCommerce catalog is an online database that showcases all the products an eCommerce
business offers in an organized and appealing manner. It includes product descriptions,
images, pricing, etc., which help customers make informed decisions. An eCommerce catalog
can be accessed through a website or mobile application. It typically includes a search
function and filters to help customers find products that meet their needs. It has features such
as product reviews and ratings and related product suggestions.
A well-designed and user-friendly eCommerce product catalog is essential for any online
business, as it can help attract and retain customers and increase revenue. Managing digital
eCommerce catalogs is a continuous process that involves organizing products in a specific
manner to maintain uniformity and quality data across sales channels. An online catalog is a
repository of product information that includes the product name, description, price, supplier,
and other relevant details.
This helps businesses reach a wider audience, reduce operational expenses, and provide
consumers the convenience of shopping from home. Additionally, consumers can compare
products and prices across different brands. The key objective of eCommerce catalog
management is to maintain consistency in product data and messaging across all sales
channels. As a crucial customer-centric sales strategy, effective catalog management for
eCommerce can significantly impact the success of online businesses.
Essential Goals for Ecommerce Catalogs
Most of your business depends on how and where you sell your products or services. Here is
how you can achieve the goal of higher sales through cataloging.
So, some tips for your e-commerce cataloging services are as follows.
1. Starting with an Online Presence
Starting and increasing online presence is essential for any business to reach a wider audience
and boost sales. Here are some steps to increase online presence:
1. Build a user-friendly website: Create a website that is easy to navigate, visually
appealing, and optimized for search engines.
2. Optimize for search engines: Use relevant keywords throughout your website content,
meta descriptions, and tags. This can help improve your website’s ranking on search
engine results pages.
3. Create high-quality content: Produce high-quality content relevant to your target
audience. This can include blog posts, videos, PowerPoint slides, infographics, and
more.
4. Use social media: Build a strong presence on social media platforms like Facebook,
Twitter, LinkedIn, and Instagram. Share engaging content and interact with your
followers.
5. Invest in paid advertising: Consider using paid advertising on search engines, social
media, and other relevant platforms to reach a wider audience.
6. Build backlinks: Build high-quality backlinks from other reputable websites to your
website. This can improve your website’s ranking on search engines and increase
your online visibility.
7. Monitor website traffic: Use website analytics tools to monitor website traffic, user
behavior, and other vital metrics. This can help you make data-driven decisions to
improve your online presence.
2. Do not sell everything, anywhere
While you launch available products, you must ensure that you sell them at the desired
marketplace that suits your business. Since you need to pay a commission to sell your
products online, you must showcase the best products you are sure about. This will create a
great impression on potential customers and limit irrelevant content that might tarnish your
brand’s image.
3. Categorizing your Products
One of the best ways to increase your sales while dealing with ecommerce cataloging is
categorizing the related items in one place. And providing alternative insights to the
customers while searching for a product.
For example, many sellers on Amazon try to show you related items when you search for any
product. Say, you are looking for a wrist watch. Relevant items will appear as you open the
product to read the specifications (different watches – similar to the one you are looking for),
allowing you to choose the most relevant.
Even Amazon has picked up this seemingly effective strategy to show related products that
are usually brought together. For example, suppose someone searches for a badminton
racquet on your marketplace/eCommerce website. In that case, it will show them a pack of
shuttlecocks alongside the racquet in the ‘Frequently brought together’ section, which makes
buyers realize the requirement of shuttlecocks when they buy the racquet, helping you sell
another product.
4. Automate your Ecommerce Catalog Management
Getting online exclusively means saving time, having a robust online presence, and vigorous
brand awareness. To expand your product base, make sure you automate your marketplace
strategically.
There are plenty of tools and solutions to help you manage your ecommerce online catalog
and enable you to steer your inventory. Ecommerce product catalog management through
such software and tools becomes exponentially more convenient.
Ecommerce catalog software such as Sellbrite, ChannelAdvisor, Pattern, CedCommerce, and
GeekSeller allows you to work with multiple shopping carts and popular marketplaces like
Amazon, Shopify, eBay, Alibaba, etc. It lets you easily synchronize the database and manage
inventory across the sales channels. It saves your time to micromanage your products and
allows you to have centralized control over inventory.
These software tools will allow you to:
1. Manage multi-channel listings
2. Focus on direct sales
3. Automate repricing
4. Improve online presence
Tip: While deciding on product pricing, offer freebies on product delivery. The visitor will
likely purchase the product. According to a study by PayPal and ComScore, 72 percent of
users will not buy the product if the shipping prices are high.
5. Make it User Friendly
The quality of your catalog determines the visibility of your products. So, make sure the
pages of your ecommerce catalogs and directories are user-friendly and engaging. We
recommend you have the following features in your digital product catalogs:
1. The user-friendly layout of your pages
2. A relevant and attractive title
3. Appealing 360-degree professional images
4. Product Category
5. Relevant Keywords
6. Using Specific Keywords
How do you get your product on the first page of every related web search?
Keeping yourself updated with the latest Marketing Trends, using specific keywords, and
optimizing your content in an easily indexable way can help you reach out to the right
audience.
Say you are selling camping gear. You want to reach out to the users searching for things like
tents, backpacks, airbeds, and other items in your stock. To achieve this, you must optimize
your website and use specific keywords to attract buyers.
Now the point is that you are not just offering these products. You need to sell them.
Therefore, using keywords like ‘tents, airbeds, backpacks’ won’t do your job. To get more
traffic, you must use specific phrases or keywords to lure the customer. That’s where SEO –
Search Engine Optimization – comes in. Your products should appear not just in the
marketplace search, but also in Google (other search engines) search result pages. This will
ensure maximum traffic flow to your product pages, increasing your sales potential.
According to Moz, Longtail Keywords constitute over 70 percent of online searches. Using
phrases like ‘all-weather tent for 4 people,’ ‘2 Person Camping & Hiking Tent, Under Rs X,
tend‘, etc., in product descriptions can help you achieve your goal of being easily searchable.
7. Use Professional Product Images
Professional product images are essential for several reasons:
1. Attracting attention: Professional photos can help your products stand out in a
crowded online marketplace, drawing more attention from potential customers.
2. Building trust: High-quality product images can help build trust with your customers
by showcasing your products in the best possible light.
3. Providing information: Professional images can help customers understand the
features and benefits of your products more clearly, providing the information they
need to make informed purchasing decisions.
4. Increasing sales: Studies have shown that high-quality product images can increase
conversion rates and sales, as customers are more likely to buy when they can see and
understand the product clearly.
5. Branding: Professional images can help reinforce your brand identity and message,
creating a consistent and recognizable image for your business.
Additional Tips for Ecommerce Catalogs
Here are some additional tips that you should keep in mind when creating digital catalogs for
your online retail business:
1. Keep the layout simple and elegant – Use uniform font, style, size, and color.
2. Page and product alignment – You should sort your products as per their quality.
Place your best-selling products on your catalog page’s right or left side.
3. Cross-selling – Cross-sell your products by placing similar products together.
4. Cross-channel marketing – Use multiple channels to reach out to your potential
buyers, say through an email, to increase your engagement and brand awareness by
166%.
5. FAQs – Answer some Q&As that the consumers frequently ask about the product. It
helps them to make an informed purchase.
6. Pictures are everything – With product description in your catalog, provide multiple
HD images with a uniform background to attract the eyeballs of the viewer that drive
him to click the “Buy Now” button.
Automate the Cataloging with Spyne
Spyne offers Ecommerce photography services to help businesses create high-quality product
images that are visually appealing and engaging. Spyne’s artificial intelligence uses its
expertise and training in millions of images to automate the editing process of product images
that showcase your products in the best possible light.
Many eCommerce photography services are offered, including product photography, 360-
degree product photography, lifestyle photography, and more. Spyne’s API also provides
image editing services to ensure your product images are flawless, consistent, and optimized
for eCommerce platforms. Some more benefits for your image editing at Spyne includes:
1. Taking care of lighting and angles
2. Removing watermarks
3. Color Correction
4. Aligning the product centrally
5. Background replacement
6. Adding margins
Information Filtering:
An information filtering system is a system that removes redundant or
unwanted information from an information stream using (semi)automated or computerized
methods prior to presentation to a human user. Its main goal is the management of
the information overload and increment of the semantic signal-to-noise ratio. To do this the
user's profile is compared to some reference characteristics. These characteristics may
originate from the information item (the content-based approach) or the user's social
environment (the collaborative filtering approach).
Whereas in information transmission signal processing filters are used against syntax-
disrupting noise on the bit-level, the methods employed in information filtering act on the
semantic level.
The range of machine methods employed builds on the same principles as those
for information extraction. A notable application can be found in the field of email spam
filters. Thus, it is not only the information explosion that necessitates some form of filters,
but also inadvertently or maliciously introduced pseudo-information.
On the presentation level, information filtering takes the form of user-preferences-
based newsfeeds, etc.
Recommender systems and content discovery platforms are active information filtering
systems that attempt to present to the user information items
(film, television, music, books, news, web pages) the user is interested in. These systems add
information items to the information flowing towards the user, as opposed to removing
information items from the information flow towards the user. Recommender systems
typically use collaborative filtering approaches or a combination of the collaborative filtering
and content-based filtering approaches, although content-based recommender systems do
exist.
History
Before the advent of the Internet, there are already several methods of filtering information;
for instance, governments may control and restrict the flow of information in a given country
by means of formal or informal censorship.
On the other hand, we are going to talk about information filters if we refer to newspaper
editors and journalists when they provide a service that selects the most valuable information
for their clients, readers of books, magazines, newspapers, radio listeners and TV viewers.
This filtering operation is also present in schools and universities where there is a selection of
information to provide assistance based on academic criteria to customers of this service, the
students. With the advent of the Internet it is possible that anyone can publish anything he
wishes at a low-cost. In this way, it increases considerably the less useful information and
consequently the quality information is disseminated. With this problem, it began to devise
new filtering with which we can get the information required for each specific topic to easily
and efficiently.
Operation
A filtering system of this style consists of several tools that help people find the most
valuable information, so the limited time you can dedicate to read / listen / view, is correctly
directed to the most interesting and valuable documents. These filters are also used to
organize and structure information in a correct and understandable way, in addition to group
messages on the mail addressed. These filters are essential in the results obtained of
the search engines on the Internet. The functions of filtering improves every day to get
downloading Web documents and more efficient messages.
Criterion
One of the criteria used in this step is whether the knowledge is harmful or not, whether
knowledge allows a better understanding with or without the concept. In this case the task
of information filtering to reduce or eliminate the harmful information with knowledge.
Learning System
A system of learning content consists, in general rules, mainly of three basic stages:
1. First, a system that provides solutions to a defined set of tasks.
2. Subsequently, it undergoes assessment criteria which will measure the performance of
the previous stage in relation to solutions of problems.
3. Acquisition module which its output obtained knowledge that are used in the system
solver of the first stage.
Future
Currently the problem is not finding the best way to filter information, but the way that
these systems require to learn independently the information needs of users. Not only because
they automate the process of filtering but also the construction and adaptation of the filter.
Some branches based on it, such as statistics, machine learning, pattern recognition and data
mining, are the base for developing information filters that appear and adapt in base to
experience. To carry out the learning process, part of the information has to be pre-filtered,
which means there are positive and negative examples which we named training data, which
can be generated by experts, or via feedback from ordinary users.
Error
As data is entered, the system includes new rules; if we consider that this data can generalize
the training data information, then we have to evaluate the system development and measure
the system's ability to correctly predict the categories of new information. This step is
simplified by separating the training data in a new series called "test data" that we will use to
measure the error rate. As a general rule it is important to distinguish between types of errors
(false positives and false negatives). For example, in the case on an aggregator of content for
children, it doesn't have the same gravity to allow the passage of information not suitable for
them, that shows violence or pornography, than the mistake to discard some appropriated
information. To improve the system to lower error rates and have these systems with learning
capabilities similar to humans we require development of systems that simulate human
cognitive abilities, such as natural-language understanding, capturing meaning Common and
other forms of advanced processing to achieve the semantics of information.
Fields of use
Nowadays, there are numerous techniques to develop information filters, some of these reach
error rates lower than 10% in various experiments. [citation needed] Among these techniques there
are decision trees, support vector machines, neural networks, Bayesian networks, linear
discriminants, logistic regression, etc.. At present, these techniques are used in different
applications, not only in the web context, but in thematic issues as varied as voice
recognition, classification of telescopic astronomy or evaluation of financial risk.
Multimedia and Digital video:
It focuses on the type of multimedia required for ecommerce applications, compression techniques,
how to store this multimedia Ex: CD-ROM. This unit also focuses on how to implement video
conferencing.
Key Multimedia concepts
, Digital Video & Electronic Commerce, Desktop Video Processing, Desk top Video
Conferencing.
Compression methods include:
Sector-oriented disk compression: It is integrated into the operating system; this
compression is invisible to end user. Ex: Compression of help files in Windows 3.1
Backup or archive-oriented compression: Compresses file before they are downloaded over
telephone lines
Graphic & video-oriented compression: Compress graphics & video file before they are
downloaded
Compression of data being transmitted over low-speed network: Techniques used in
modems, routers to send data over low bandwidth phone lines.
