VIDEO INDEXING AND SEARCHING BASED ON
AUDIO TEXT ANALYSIS
Abstract - Vision-based
analysis is extremely difficult due to the various concepts (object, action,
and scene) contained in videos. Visual concept-based analysis has achieved
significant progress. We extract metadata from visual as well as audio
resources of videos automatically by applying appropriate analysis
techniques.To deal with these issues, we propose a automatic visual concept
learning algorithm for event understanding in videos. For evaluation purposes
we developed a several automatic indexing functionalities in a large video
portal, which can guide both visually and text-oriented users to navigate
within video. The extensive experimental evaluations on the collected dataset.
We conducted a user study which intended to verify the research hypothesis and
to investigate the usability and the effectiveness of proposed.
Index terms: video indexing, video recognition,
audio analysis.
I. INTRODUCTION
We
are in age often referred to as the information age. In this information age,
because we believe that information leads to power and success, and thanks to
sophisticated technologies such as computers, satellites, etc., we have been
collecting tremendous amounts of information. Initially, with the advent of
computers and means for mass digital storage, we started collecting and storing
all sorts of data, counting on the power of computers to help sort through this
amalgam of information. In this project, we take a
Video/Audio recording and use Sphinx framework to split audio from the video
source, once we split the audio from video source, we parse the audio using
sphinx framework and extract text from it. We also capture the time in which
these text are spoken. Using this information we index the transcriptions we
refer this approach as chaining. And offer user to search lecture contents and
find at what time frame the specific transcriptions appear on the video, and
play video from that time frame. Clustering is used to find cohesive areas; the word stream from
audio file can contain raw data of all parts of speech. We create a chain from
the raw data which has all words from first to last appearance of the term in
the lecture. The chain is a accumulation of equal terms. There are two main
steps involved in this process. First video is split into coherent segments and
secondly the topics have to specify the topic with descriptions. Once the words
are chained along with the time frame it appear, an application is built to
search the indexed words. Where this app allows user to search for the index..It
is the crucial step in which clever techniques are applied to extract patterns
potentially useful. Data Mining, also popularly known as Knowledge
Discovery in Databases (KDD), refers to the nontrivial extraction of
implicit, previously unknown and potentially useful information from data in
databases. While data mining and knowledge discovery in databases (or KDD) are
frequently treated as synonyms, data mining is actually part of the knowledge
discovery process.
II. RELATED WORK
While
preparing recorder lectures for content based retrieval the automated indexing
of multimedia videos and the retrieval of semantically appropriate information
from a lecture knowledge base. It is evident that the rapid growth of
multimedia data available in e-learning system requires more efficient method
for content-based browsing and retrieval of video data. The indexer adds the
referenced document to the document list for the appropriate words. In the
larger search engine, the process of finding each word in the inverted index(in
order to report that it occurred within a document)may be two times consuming ,
so this process is commonly split up into two parts, the development of a forward
index and a process which sorts the contents of the forward into the inverted
index. The inverted index is so named because it is an inversion of the forward
index. The requested information is often covered by only a few minutes of the
lecture recording and is therefore hidden within a full ninety minutes a
recording stored among 1000s of others. It is often not the problem to find the
proper lecture in the achieve but rather than to find the proper position
inside the video stream. It is not practical for learners to watch the whole
video to get the desired information inside the lecture video. The problem
becomes how to retrieve the appropriate information in a large lecture video database,
more efficiently.
III. SYSTEM DESIGN
Video
indexing is a data structure technique to efficiently retrieve record from the
databases files based on some attribute on which the indexing has been done.
The preview diagram helps you cut the particular part of a video files to
convert. You can set the start and end time by dragging the slider bar .
Checking the cut clip and then you can get the accurate audio part of the
converted file.Now you didn’t need to convert a whole file of several hours
long just to get a piece of music of several seconds. Video indexing is used to retrieve the data.
Video provides both frames and images. Its separates frames and images by
boosted concept algorithm. The above diagram describes about Sphinx framework.
This concept provides user to search a text for easy understanding of video. User
provides the text for search. Meanwhile the video of all the frames are
converted into audio files. This audio is converted into text file by the
concept of algorithm Sphinx framework.
Fig1.System Design.
The
purpose of sphinx framework is to convert video speech into text files. This
method compares text of both video and user. This provides the series of
options in which user wants to select a particular frames. After selecting a
particular frame the video will started from the given text. The video indexing
providers a less time to search a word from video frame. Final result will be
displayed within few seconds.
Fig2.
Functional diagram
Video
file is converting into audio files or voice. The audio file is uploaded. The
application of Sphinx framework is used to convert video files into text files.
Its provides request or response. If the given text files provider the correct
word the video file will be displayed otherwise is display the error
correction. Then the user can view the video from the particular text files.
Videoindexing
and searching based on audio text analysis is spitted into three modules.
A.Converting
the video into audio.
B. Extracting
the data from the audio.
C. Indexing
the texts from the audio.
Converting the video into audio.
The
video file is processed into frames and images. While processing the video
files is converted into audio. This separates the video into frames and images.
Extracting the data from the audio.
From
the audio the data is extracted for the audio text analysis. This method
provides better searching of text in the video files.
Indexing the text from the audio.
Sphinx
framework is used to extract data from audio. The purpose of Sphinx framework
is to convert video speech into text files. Indexing is used to retrieve the
data.
IV.SPHINX FRAMEWORK
Sphinx is a tool that makes it easy to create
intelligent and beautiful documentation, written by Georg Brandi and licensed
under the BSD license. Here are few Sphinx Unique Features: high indexing speed
(up to 10 MB/sec on modern CPUs)high search(avg query is under 0.1 sec on 2-4
GB text collections)high scalability (up to 100 GB of text, upto100Mdocuments
on a single CPU)provides distributed searching capabilitiesprovides searching
from within MYSQL through pluggable storage enginesupportsBoolean, phrase, and
word proximity queriessupports multiple full-text fields per document (upto32
by default)supports multiple additional attributes per document (ie. groups,
timestamps, etc)supports MYSQL natively (MYISAM and InnoDB tables are both
supported).
A. METHODOLOGY
The
methodology used here are Sphinx framework and boosted concept. In practice ,
some images frames are not truly related to the concepts assigned to a video.
These image will add too much noise if we learn concept classifiers using the
extracted visual features from the whole video.
Fig3. Accuracy
result of boosted concept algorithm using boosting framework.
In
this section, we introduce a boosted concept learning algorithm to iteratively
obtain multiple classifiers for each concept. With the help of auxiliary web
images, our concept classifiers are trained using the most related images in
video. We first overview the whole process of our algorithm.
B.Feature
High indexing speed(up
to 10MB/sec on modern CPUs), High search speed(avg query is under 0.1 sec on
2-4 GB text collections), High scalability(up to 100 GB of text, up ti 100 M
documents on a single CPU). Provider distributed searching capabilities.Then
provides searching from within mysql through pluggable storage engine support
the Boolean,phase ,and word proximity queries support multiple full-text fields
per document(up to 32 by default) support multiple additional attributes per
document(ie. Grouping, timestamps, etc) support mySql natively (my ISAM and
INNODB tables are both supported). The scope for future extension as a
enhancement this application can recognize local languages to provide better
sale information to the user.
Additionally the
application can also be enhanced to recognize user written script in various
local languages.
V. CONCLUSION
In
this paper, we presented an approach for content-based lecture video indexing
and retrieval in large lecture video archives. Speech recognition technology
has a wide range of applications in learning system from captioning video,
television for the hearing-impaired, voice controlled computer operation, till
dictation. Some of the most popular commercially available applications of
Speech Recognition are for dictation and other hands-free writing tasks with software
applications. The commercial SR tools are commonly said to achieve 98% accuracy
but for the spontaneous speech the accuracy cannot be achieved in the same way
due to number of reasons .Digital video recording and
learning from video lectures has increased dramatically. A lot of organizations
are using video lectures to guide project teams after hours. It is not easy to
find an specific information from the specific recording since the requested
information will be covered only by few minutes of the lecture recording and
therefore it is hidden within a full duration of the lecture or meeting.
Detailed browsing in video is not supported due to lack of explicit annotation,
manual annotation and segmentation is time consuming.
VI. FEATURE ENHANCEMENT
As
an enhancement this application can recognize local languages (Tamil, Hindi
etc) to provide better sale information to the user. Additionally the
application can also be enhanced to recognize user written scripts in various
local languages. We can also insert image as an input and get the required
video files.
REFERENCES
[1] B.-K. Bao, W. Min, K. Lu, and C. Xu, “Social event detection
with robust high-order co-clustering,” in Proc. 3rd ACM Int. Conf. Multi- media
Retrieval, 2013, pp. 135–142.
[2] M. Zaharieva, M. Zeppelzauer, and C. Breiteneder, “Automated
social event detection in large photo collections,” in Proc. ACM Int. Conf.
Multimedia Retrieval, 2013, pp. 167–174.
[3]G.Petkos,S.Papadopoulos,andY.Kompatsiaris,“Social eventdetection
using multimodal clustering and integrating supervisory signals,” in Proc. 2nd
ACM Int. Conf. Multimedia Retrieval, New York, NY, USA, 2012, pp. 23:1–23:8.
[4] M. Brenner and E. Izquierdo, “Social event detection and
retrieval in collaborative photo collections,” in Proc. 2nd ACM Int. Conf.
Multi- media Retrieval, 2012, pp. 21:1–21:8.
[5] Y. Wang, H. Sundaram, and L. Xie, “Social event detection with
in- teraction graph modeling,” in Proc. ACM Int. Conf. Multimedia,2012, pp.
865–868.
[6] J. Liu, Q. Yu, O. Javed, S. Ali, A. Tamrakar, A.Divakaran,H.Cheng,
andH.S.Sawhney,“Videoeventrecognitionusingconceptattributes,” in Proc. IEEE
Workshop Appl. Comput. Vis., Jan. 2013, pp. 339–346.
[7]. Z. Ma, Y. Yang, Z. Xu, S. Yan, N. Sebe, and A. G. Hauptmann,
“Com- plex event detection via multi-source video attributes,” in Proc. IEEE
Conf. Comput. Vis. Pattern Recog., Jun. 2013, pp. 2627–2633.
[8]Y.Yang,Z.Ma,Z.Xu,S.Yan,andA.G.Hauptmann,“Howrelatedex- emplars
help complex event detection in web videos?,” in Proc. IEEE Int. Conf. Comput.
Vis., Dec. 2013, pp. 2104–2111.
[9] Z. Ma, Y. Yang, Z. Xu, N. Sebe, and A. G. Hauptmann, “We are
not equally negative: Fine-grained labeling for multimedia event detec- tion,”
in Proc. ACM Int. Conf. Multimedia, 2013, pp. 293–302.
[10] Q. Yu, J. Liu, H. Cheng, A. Divakaran, and H. S. Sawhney,
“Multi- media event recounting with concept based representation,” in Proc. ACM
Conf. Multimedia, 2012, pp. 1073–1076.
