Adwatch
![](https://calbyte.us/wp-content/uploads/2024/05/ad2.png)
![](https://calbyte.us/wp-content/uploads/2024/05/ad.png)
AdWatch is a complete workflow for automatic audio and video content detection, including advertisements and promos.
Stack (OS & PL & FW):
Operating System:
Linux
Programming Language:
Python Framework:
Audfprint, ffmpeg, moviepy
Dataset:
Image dataset of advertisements and news domain .wav files of advertisements for database generation Database containing audio fingerprints of advertisements in a pickle file.
Workflow (Processing):
Video file (.mp4) to audio file (.wav) conversion with standard sample and bitrate Detecting ads by matching their fingerprints from the input file Additional frames detection using CNN Removal of ads from the original input (.mp4) file Output is .mp4 with removed advertisements.
End To End (Development & Integration in a System):
Step1 (Dataset Generation):
For Transfer Learning (VGG-16): .mp4 files of news domain and advertisements with 50:50 or 60:40 time ratio
Frames extraction of both classes
Data annotation
Pre-processing Images according to VGG-16 model
For Audfprint
.mp4 files of advertisements
Conversion of .mp4 file into .wav files with the standard sample rate of 44.1 kHZ and bitrate of 1411k.
Step2 (Training):
Transfer Learning (VGG-16 Model):
Fine-tuned VGG-16 according to our problem and number of classes
Freezed all the layers except the last four layers, which were re-trained for the feature extraction process.
Last output layer neurons were set to 2, as our problem is binary classification.
After training, the best model was saved.
Audfprint:
Generated database of audio fingerprints using following command: ‘python audfprint.py new –dbase database_name.pklz path_of_audio_file/*.mp3 or .wav’
Step3 (Fusion of Audfprint and VGG-16):
Input .mp4 file containing news and ads
Conversion of .mp4 file to .wav file.
Matching maximum ads existing in the file using Audfprint
Updating the time range given by Audfprit through CNN model.
CNN is used to predict the ads and news frames prior to starting time and after the ending time of detected ads.
Removal of ads using starting and ending time.
Output is .mp4 file with all the ads removed.
Step4 (Testing):
Evaluation of Adwatch by feeding in a video stream of news with ads.
Deployment (Server / API):
Output (Screenshots):
Applications (General Real World Use):
Audio and video content detection
Ads verification
Competitive ads analysis
Ads Affidavits reports
Use Case (Our Specific):
To efficiently monitor all media news campaigns i.e., news bulletins, talk shows, press conferences, headlines etc., by removing news commercials that come in the way.
Task
Artificial Intelligence