- Video Database for your AI Applications @@ -34,8 +34,8 @@
-# VideoDB Python Client -The VideoDB Python client is a python package that allows you to interact with the VideoDB which is a serverless database that lets you manage video as intelligent data, not files. It is secure, scalable & optimized for AI- applications and LLM integrations. +# VideoDB Python SDK +VideoDB Python SDK allows you to interact with the VideoDB serverless database. Manage videos as intelligent data, not files. It's scalable, cost efficient & optimized for AI applications and LLM integration. ## Quick Start ### Creating a Connection -To create a new connection you need to get API key from [VideoDB console](https://console.videodb.io). You can directly upload from youtube, any public url, S3 bucket or local file path. A default collection is created when you create a new connection. +Get API key from [VideoDB console](https://console.videodb.io). Free for first 50 uploads. _(No credit card required)_ ```python import videodb - -# create a new connection to the VideoDB conn = videodb.connect(api_key="YOUR_API_KEY") +``` +## Working with a single Video -# upload to the default collection using the video url returns a Video object -video = conn.upload(url="https://www.youtube.com/") +--- -# upload to the default collection using the local file path returns a Video object -video = conn.upload(file_path="path/to/video.mp4") +β¬οΈ **Uploading a Video** -# get the stream url for the video -stream_url = video.get_stream() +Now that you have established a connection to VideoDB, you can upload your videos using `conn.upload()` +You can directly upload from `youtube`, `any public url`, `S3 bucket` or `local file path`. A default collection is created when you create a new connection. -``` - -### Getting a Collection -To get a collection, use the `get_collection` method on the established database connection object. This method returns a `Collection` object. +`upload` method returns a `Video` object. ```python -import videodb +# Upload a video by url +video = conn.upload(url="https://www.youtube.com/watch?v=WDv4AWk0J3U") -# create a connection to the VideoDB -conn = videodb.connect(api_key="YOUR_API_KEY") +# Upload a video from file system +video_f = conn.upload(file_path="./my_video.mp4") + +``` -# get the default collection -collection = conn.get_collection() +### πΊ Viewing your video -# Upload a video to the collection returns a Video object -video = collection.upload(url="https://www.youtube.com/") +Your video is instantly available for viewing 720p resolution β‘οΈ -# async upload -collection.upload(url="https://www.youtube.com/", callback_url="https://yourdomain.com/callback") +* Generate a streamable url for video using video.generate_stream() +* Preview the video using video.play(). This will open the video in your default browser/notebook -# get all the videos in the collection returns a list of Video objects -videos = collection.get_videos() +```python +video.generate_stream() +video.play() +``` -# get a video from the collection returns a Video object -video = collection.get_video("video_id") +### βοΈ Stream Sections of videos -# delete the video from the collection -collection.delete_video("video_id") +You can easily clip specific sections of a video by passing timeline of start and end sections. +It accepts seconds. For example, Hereβs we are streaming only first `10 seconds` and then `120` to `140 second` of a video +```python +stream_link = video.generate_stream(timeline=[[0,10], [120,140]]) +play_stream(stream_link) ``` -### Multi Modal Indexing +### π Searching inside a video -#### Spoken words indexing +To search bits inside a video β you have to index the video first. This can be done by a simple command. +_Indexing may take some time for longer videos._ ```python -import videodb +video.index_spoken_words() +result = video.search("Morning Sunlight") +result.play() +video.get_transcript() +``` +`Videodb` is launching more indexes in upcoming versions. +Currently it offers semantic index - Index by spoken words. -# create a connection to the VideoDB and get the default collection -conn = videodb.connect(api_key="YOUR_API_KEY") -collection = conn.get_collection() +In future you can also index videos using: +1. **Scene** - Visual concepts and events. +2. **Faces**. +3. **Specific domain Index** like Football, Baseball, Drone footage, Cricket etc. -# get the video from the collection -video = collection.get_video("video_id") +### Viewing Search Results : -# index the video for symantic search -video.index_spoken_words() +`video.search()` will return a `SearchResults` object, which contains the sections/shots of videos which semantically match your search query -# search relevant moment in video and stream resultant video clip instantly. -# returns a SearchResults object -# for searching the video, the video must be indexed please use index_spoken_words() before searching -# optional parameters: -# - type: Optional[str] to specify the type of search. default is "semantic" -# - result_threshold: Optional[int] to specify the number of results to return. default is 5 -# - score_threshold: Optional[float] to specify the score threshold for the results. default is 0.2 -result = video.search("what is videodb?") -# get stream url of the result -stream_url = result.compile() -# get shots of the result returns a list of Shot objects -shots = result.get_shots() -# get stream url of the shot -short_stream_url = shots[0].compile() - -# search relevant moment in collections and stream resultant video clip instantly. -# returns a SearchResults object -result = collection.search("what is videodb?") -# get stream url of the result -stream_url = result.compile() -# get shots of the result returns a list of Shot objects -shots = result.get_shots() -# get stream url of the shot -short_stream_url = shots[0].get_stream() +* `result.get_shots()` Returns a list of Shot that matched search query +* `result.play()` Returns a playable url for video (similar to video.play() you can open this link in browser, or embed it into your website using iframe) -``` +## RAG: Search inside Multiple Videos + +--- + +`VideoDB` can store and search inside multiple videos with ease. By default, videos are uploaded to your default collection. + +### π Using Collection to upload multiple Videos -### Video Object Methods ```python -import videodb +# Get the default collection +coll = conn.get_collection() -# create a connection to the VideoDB, get the default collection and get a video -conn = videodb.connect(api_key="YOUR_API_KEY") -collection = conn.get_collection() -video = collection.get_video("video_id") +# Upload Videos to a collection +coll.upload(url="https://www.youtube.com/watch?v=lsODSDmY4CY") +coll.upload(url="https://www.youtube.com/watch?v=vZ4kOr38JhY") +coll.upload(url="https://www.youtube.com/watch?v=uak_dXHh6s4") +``` +* `conn.get_collection()` : Returns Collection object, the default collection +* `coll.get_videos()` : Returns list of Video, all videos in collections +* `coll.get_video(video_id)`: Returns Video, respective video object from given `video_id` +* `coll.delete_video(video_id)`: Deletes the video from Collection -# get the stream url of the dynamically curated video based on the given timeline sequence -# optional parameters: -# - timeline: Optional[list[tuple[int, int]] to specify the start and end time of the video -stream_url = video.get_stream(timeline=[(0, 10), (30, 40)]) +### π Search inside collection -# get thumbnail of the video -thumbnail = video.get_thumbnail() +You can simply Index all the videos in a collection and use +search method on collection to find relevant results. +Here we are indexing spoken content of a +collection and performing semantic search. +```python +# Index all videos in collection +for video in coll.get_videos(): + video.index_spoken_words() -# get transcript of the video -# optional parameters: -# - force: Optional[bool] to force get the transcript. default is False -transcript = video.get_transcript() +# search in the collection of videos +results = coll.search(query = "What is Dopamine?") +results.play() +``` +The result here has all the matching bits in a single stream from your collection. You can use these results in your application right away. + +### π Explore the Video object -# get transcript text of the video -# optional parameters: -# - force: Optional[bool] to force get the transcript text. default is False -transcript_text = video.get_transcript_text() +There are multiple methods available on a Video Object, that can be helpful for your use-case. -# add subtitle to the video and get the stream url of the video with subtitle -stream_url = video.add_subtitle() +**Access Transcript** +```python +# words with timestamps +text_json = video.get_transcript() +text = video.get_transcript_text() +print(text) +``` -# delete the video from the collection -video.delete() +**Add Subtitle to a video** +It returns a new stream instantly with subtitle added into the video. +```python +new_stream = video.add_subtitle() +play_stream(new_stream) ``` +**Get Thumbnail of Video:** - -## Roadmap +`video.generate_thumbnail()`: Returns a thumbnail image of video. -See the [open issues](https://github.com/video-db/videodb-python/issues) for a list of proposed features (and known issues). +**Delete a video:** +`video.delete()`: Delete a video. +Checkout more examples and tutorials π [Build with VideoDB](https://docs.videodb.io/build-with-videodb-35) to explore what you can +build with `VideoDB` + +--- + +## Roadmap +- Adding More Indexes : `Face`, `Scene`, `Security`, `Events`, and `Sports` +- Give prompt support to generate thumbnails using GenAI. +- Give prompt support to access content. +- Give prompt support to edit videos. +- See the [open issues](https://github.com/video-db/videodb-python/issues) for a list of proposed features (and known issues). + +--- ## Contributing @@ -195,11 +215,7 @@ Contributions are what make the open source community such an amazing place to b 4. Push to the Branch (`git push origin feature/AmazingFeature`) 5. Open a Pull Request - -## License - -Distributed under the MIT License. See `LICENSE` for more information. - +--- diff --git a/setup.py b/setup.py index 6756694..86ef2b4 100644 --- a/setup.py +++ b/setup.py @@ -21,9 +21,9 @@ def get_version(): setup( name="videodb", version=get_version(), - author="Videodb", + author="videodb", author_email="contact@videodb.io", - description="Videodb Python client", + description="VideoDB Python SDK", long_description=long_description, long_description_content_type="text/markdown", url="https://github.com/video-db/videodb-python", @@ -33,5 +33,9 @@ def get_version(): "requests>=2.25.1", "backoff>=2.2.1", ], - + classifiers=[ + "Intended Audience :: Developers", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.9", + ], ) diff --git a/videodb/__init__.py b/videodb/__init__.py index 02c7fa5..b5f18f4 100644 --- a/videodb/__init__.py +++ b/videodb/__init__.py @@ -4,6 +4,7 @@ import logging from typing import Optional +from videodb._utils._video import play_stream from videodb._constants import VIDEO_DB_API from videodb.client import Connection from videodb.exceptions import ( @@ -15,7 +16,7 @@ logger: logging.Logger = logging.getLogger("videodb") -__version__ = "0.0.1" +__version__ = "0.0.2" __author__ = "videodb" __all__ = [ @@ -23,6 +24,7 @@ "AuthenticationError", "InvalidRequestError", "SearchError", + "play_stream", ] diff --git a/videodb/_constants.py b/videodb/_constants.py index fca23af..9911271 100644 --- a/videodb/_constants.py +++ b/videodb/_constants.py @@ -4,6 +4,7 @@ VIDEO_DB_API: str = "https://api.videodb.io" + class SearchType: semantic = "semantic" diff --git a/videodb/_utils/_http_client.py b/videodb/_utils/_http_client.py index 0c30c66..07a19c1 100644 --- a/videodb/_utils/_http_client.py +++ b/videodb/_utils/_http_client.py @@ -120,7 +120,7 @@ def _handle_request_error(self, e: requests.exceptions.RequestException) -> None f"Invalid request: {str(e)}", e.response ) from None - @backoff.on_exception(backoff.expo, Exception, max_time=500) + @backoff.on_exception(backoff.expo, Exception, max_time=500, logger=None) def _get_output(self, url: str): """Get the output from an async request""" response_json = self.session.get(url).json() diff --git a/videodb/_utils/_video.py b/videodb/_utils/_video.py new file mode 100644 index 0000000..9cdb012 --- /dev/null +++ b/videodb/_utils/_video.py @@ -0,0 +1,22 @@ +import webbrowser as web +PLAYER_URL: str = "https://console.videodb.io/player" + + +def play_stream(url: str): + """Play a stream url in the browser/ notebook + + :param str url: The url of the stream + :return: The player url if the stream is opened in the browser or the iframe if the stream is opened in the notebook + """ + player = f"{PLAYER_URL}?url={url}" + opend = web.open(player) + if not opend: + try: + from IPython.display import IFrame + + player_width = 800 + player_height = 400 + return IFrame(player, player_width, player_height) + except ImportError: + return player + return player diff --git a/videodb/search.py b/videodb/search.py index 6543793..f42f645 100644 --- a/videodb/search.py +++ b/videodb/search.py @@ -1,4 +1,5 @@ from abc import ABC, abstractmethod +from videodb._utils._video import play_stream from videodb._constants import ( SearchType, ApiPath, @@ -15,8 +16,8 @@ class SearchResult: def __init__(self, _connection, **kwargs): self._connection = _connection self.shots = [] - self.text_summary = None - self.stream = None + self.stream_url = None + self.player_url = None self.collection_id = "default" self._results = kwargs.get("results", []) self._format_results() @@ -38,18 +39,27 @@ def _format_results(self): ) ) + def __repr__(self) -> str: + return ( + f"SearchResult(" + f"collection_id={self.collection_id}, " + f"stream_url={self.stream_url}, " + f"player_url={self.player_url}, " + f"shots={self.shots})" + ) + def get_shots(self) -> List[Shot]: return self.shots def compile(self) -> str: - """Compile the search result shots into a stream link + """Compile the search result shots into a stream url :raises SearchError: If no shots are found in the search results - :return: The stream link + :return: The stream url :rtype: str """ - if self.stream: - return self.stream + if self.stream_url: + return self.stream_url elif self.shots: compile_data = self._connection.post( path=f"{ApiPath.compile}", @@ -62,12 +72,22 @@ def compile(self) -> str: for shot in self.shots ], ) - self.stream = compile_data.get("stream_link") - return self.stream + self.stream_url = compile_data.get("stream_url") + self.player_url = compile_data.get("player_url") + return self.stream_url else: raise SearchError("No shots found in search results to compile") + def play(self) -> str: + """Generate a stream url for the shot and open it in the default browser + + :return: The stream url + :rtype: str + """ + self.compile() + return play_stream(self.stream_url) + class Search(ABC): """Search interface inside video or collection""" diff --git a/videodb/shot.py b/videodb/shot.py index f592616..d715de0 100644 --- a/videodb/shot.py +++ b/videodb/shot.py @@ -2,6 +2,7 @@ from typing import Optional +from videodb._utils._video import play_stream from videodb._constants import ( ApiPath, ) @@ -29,7 +30,8 @@ def __init__( self.end = end self.text = text self.search_score = search_score - self.stream = None + self.stream_url = None + self.player_url = None def __repr__(self) -> str: return ( @@ -40,22 +42,23 @@ def __repr__(self) -> str: f"end={self.end}, " f"text={self.text}, " f"search_score={self.search_score}, " - f"stream={self.stream})" + f"stream_url={self.stream_url}, " + f"player_url={self.player_url})" ) def __getitem__(self, key): """Get an item from the shot object""" return self.__dict__[key] - def get_stream(self) -> str: - """Get the shot into a stream link + def generate_stream(self) -> str: + """Generate a stream url for the shot - :return: The stream link + :return: The stream url :rtype: str """ - if self.stream: - return self.stream + if self.stream_url: + return self.stream_url else: stream_data = self._connection.post( path=f"{ApiPath.video}/{self.video_id}/{ApiPath.stream}", @@ -64,5 +67,15 @@ def get_stream(self) -> str: "length": self.video_length, }, ) - self.stream = stream_data.get("stream_link") - return self.stream + self.stream_url = stream_data.get("stream_url") + self.player_url = stream_data.get("player_url") + return self.stream_url + + def play(self) -> str: + """Generate a stream url for the shot and open it in the default browser/ notebook + + :return: The stream url + :rtype: str + """ + self.generate_stream() + return play_stream(self.stream_url) diff --git a/videodb/video.py b/videodb/video.py index d639795..0a17113 100644 --- a/videodb/video.py +++ b/videodb/video.py @@ -1,3 +1,5 @@ +from typing import Optional +from videodb._utils._video import play_stream from videodb._constants import ( ApiPath, SearchType, @@ -6,7 +8,6 @@ ) from videodb.search import SearchFactory, SearchResult from videodb.shot import Shot -from typing import Optional class Video: @@ -14,10 +15,11 @@ def __init__(self, _connection, id: str, collection_id: str, **kwargs) -> None: self._connection = _connection self.id = id self.collection_id = collection_id - self.stream_link = kwargs.get("stream_link", None) + self.stream_url = kwargs.get("stream_url", None) + self.player_url = kwargs.get("player_url", None) self.name = kwargs.get("name", None) self.description = kwargs.get("description", None) - self.thumbnail = kwargs.get("thumbnail", None) + self.thumbnail_url = kwargs.get("thumbnail_url", None) self.length = float(kwargs.get("length", 0.0)) self.transcript = kwargs.get("transcript", None) self.transcript_text = kwargs.get("transcript_text", None) @@ -27,10 +29,11 @@ def __repr__(self) -> str: f"Video(" f"id={self.id}, " f"collection_id={self.collection_id}, " - f"stream_link={self.stream_link}, " + f"stream_url={self.stream_url}, " + f"player_url={self.player_url}, " f"name={self.name}, " f"description={self.description}, " - f"thumbnail={self.thumbnail}, " + f"thumbnail_url={self.thumbnail_url}, " f"length={self.length})" ) @@ -63,16 +66,16 @@ def delete(self) -> None: """ self._connection.delete(path=f"{ApiPath.video}/{self.id}") - def get_stream(self, timeline: Optional[list[tuple[int, int]]] = None) -> str: - """Get the stream link of the video + def generate_stream(self, timeline: Optional[list[tuple[int, int]]] = None) -> str: + """Generate the stream url of the video :param list timeline: The timeline of the video to be streamed. Defaults to None. :raises InvalidRequestError: If the get_stream fails - :return: The stream link of the video + :return: The stream url of the video :rtype: str """ - if not timeline and self.stream_link: - return self.stream_link + if not timeline and self.stream_url: + return self.stream_url stream_data = self._connection.post( path=f"{ApiPath.video}/{self.id}/{ApiPath.stream}", @@ -81,16 +84,16 @@ def get_stream(self, timeline: Optional[list[tuple[int, int]]] = None) -> str: "length": self.length, }, ) - return stream_data.get("stream_link") + return stream_data.get("stream_url", None) - def get_thumbnail(self): - if self.thumbnail: - return self.thumbnail + def generate_thumbnail(self): + if self.thumbnail_url: + return self.thumbnail_url thumbnail_data = self._connection.get( path=f"{ApiPath.video}/{self.id}/{ApiPath.thumbnail}" ) - self.thumbnail = thumbnail_data.get("thumbnail") - return self.thumbnail + self.thumbnail_url = thumbnail_data.get("thumbnail_url") + return self.thumbnail_url def _fetch_transcript(self, force: bool = False) -> None: if self.transcript and not force: @@ -111,7 +114,7 @@ def get_transcript_text(self, force: bool = False) -> str: return self.transcript_text def index_spoken_words(self) -> None: - """Symantic indexing of spoken words in the video + """Semantic indexing of spoken words in the video :raises InvalidRequestError: If the video is already indexed :return: None if the indexing is successful @@ -132,7 +135,7 @@ def add_subtitle(self) -> str: "type": Workflows.add_subtitles, }, ) - return subtitle_data.get("stream_link") + return subtitle_data.get("stream_url", None) def insert_video(self, video, timestamp: float) -> str: """Insert a video into another video @@ -140,7 +143,7 @@ def insert_video(self, video, timestamp: float) -> str: :param Video video: The video to be inserted :param float timestamp: The timestamp where the video should be inserted :raises InvalidRequestError: If the insert fails - :return: The stream link of the inserted video + :return: The stream url of the inserted video :rtype: str """ if timestamp > float(self.length): @@ -171,5 +174,12 @@ def insert_video(self, video, timestamp: float) -> str: for shot in all_shots ], ) - stream_link = compile_data.get("stream_link") - return stream_link + return compile_data.get("stream_url", None) + + def play(self) -> str: + """Open the player url in the browser/iframe and return the stream url + + :return: The stream url + :rtype: str + """ + return play_stream(self.stream_url)