More Input

DirectShow, that component that your Media player is built upon, consists of Filters, which you may have noticed while configuring that media player.

Each filter has one or more Input Pins and one or more Output Pins

A compressed image enters at the input pin of a "decoder filter" comes out uncompressed from the out pin to the next filter in the chain, which is called "Video Renderer Filter" which displays it, but since it has no output pin Sink it drops the image, and sends a ready message upstream to the "decoder Filter" the intermediate Transition filter, telling it to send the next image, which in turn tells the "File Filter" the Source to push another image downstream. that is called a DirectShow Graph.

OK, about twInput
we have 3 main filters in the twGraph :)
Source...
1- a Capture Filter "the camera driver" or a Video File Reader
2- any neccesary decoding/Resizing filters to make the images "Media Samples" sent understandable to the next filters
Transition..
3- a Sample Grabber , it reads the samples (images), and sends them to a callback where we can process them, you cna either have a copy of the data, or the original data that will be sent again down stream.
Sink...
4- again some decoders so the next uderstands the samples
5- a VideoRenderer "now you have your media player" or a Null Filter , it displays the images recived then Drops the allocated space for the images.
...

Each filter can have prefered media types, so when DS tries to connect an output pin with an input pin, it finds the most favourable media type, if pins dont prefer the same type, they can nevver connect, DS resolves this by inserting some filters in between "decoders/converters" in an action called "Intelligent connect"
...
All this ado is wrapped in a simple interface "start/capture/stop/readFromFile" so whenever a decision is made to support other platforms/devices it should be somehow easy "dreaming".
...
Note: we did not use ATL in order to convince DS to get programmed, maybe because even if we used it it wont look like C# com interface, "did they call that Interop ?"
or maybe because ATL is not a walk in the park, "i think"
or maybe because we didn't need to use any Events or Sinks or Monks or any of the other crazy COM stuff of which i am not aware and the ATL was built for.
just wanted to remind my self of that.

Touchwork Input

In this post i will discuss the twInput component and try to elaborate some decisions made during its development.

Facts:

1-twInput is developed from scratch over DirectShow
so why not use a library:

• Input is the main dependence of the framework, so if it is not flexible enough it will force the framework to take a certain shape, i.e. we will think "how to write TW and still fit with Input library" instead of "how to write TW", and still even when it is done we can change it any time to adapt with the changes in the main framework.
• We needed the input to come form a camera or a video file, so we can easily test the code even if the hardware is not available, and lacked in most of the libraries.
• Imagine spending a day learning a library interface, then after a month you have to edit in the library itself, so you move to learn the library interface that the library depends on , and then understand the code of the first top library dealing with the lower level library
  just too much ..

2-There are 2 main lowlevel alternatives for DirectShow
why not use them ?

• ActiveMovie, outdated Now is called DirectShow :D
  
• Video For Windows (VFW): much easier than DS , but outdated, mostly made for VxD devices which stopped production since 2000, now its WDM .. more details in the proposal

now i will move to the Design of twInput in another post

My World of Silicon ..

In a devastating search for a good compliant surface, I had to start experimenting at home. Most people abroad used Rosco sheets, Silicon sheets, milk paper, caulk paper, and even silk. Silicon sheets gave best results so far; unfortunately we don't have that here in Egypt ..

Just a reminder: Why do we need a compliant surface?

1. So "moving" your fingers over the surface becomes easier, and will provide better tracking. The motion is not usually "smooth" enough over bare acrylic. It also holds less marks or stains comparable with the acrylic.
   
2. Works as a projection surface. Projecting light on a transparent medium will show barely nothing.
3. Dims the back-light on the other side of the acrylic sheet; it's really great working in a well-lit room.
   

So, I decided to try making my own silicon sheet at home ..
I bought a silicon tube, emptied it in a tray, n left it to dry till the next day (I'll add photos ASAP).

Results:

• Silicon is so thick; spreading it evenly on a surface is nearly impossible. A solvent is needed to make it less thick, it could even be sprayed then.
• Also the thickness creates millions of air bubbles, which will cause noise later in the detection process.
• With a live test over the hardware, it's PERFECT!!! :D

I contacted two companies to help us out; one hung-up, and the other knows nothing about silicone. Therefore, the next step will be finding a solvent that will help me reduce the thickness of the silicon so I can spread it evenly across the sheet.

Hope ..

I thought of trying out some crazy experiment that might help us find a good compliant surface for the hardware. We decided to meet in college before our first meeting at ITWorx to integrate our tasks and try them out, also a chance for me to see the results of my experiment. The shock was to find that the hardware doesn't work! All circuit connections were totally messed up. The camera didn't work. It was horrible!

The camera issue was fixed a7l (!!!) The hardware will need a proper reconstruction. Fouad and Mahmoud paid El-Nekheily a visit and bought a few electronic-circuit-slim-boards to hold the LEDs. We tried one of them so far. Using them seems promising.

The good news is: My experiment worked a7l!!! (I'll discuss that in a later post isA)

After the continuous failure to fix things, we had to hurry to ITWorx. Mihan and Essam were so nice, they are so helpful as well. They both mentioned two hardware stores where we might find some parts for our hardware construction. They sent me details to check RS and Maamoun stores; we should be doing that in the coming few days, or after we're back isA.

Meeting Summary:

• Assign roles between us - to be discussed.
• Create a task-list - suggestions: Google Notebook or GoogleDocs (Excel Spreadsheet).
• Start writing the requirements document till we get the templates from ITWorx.
  
• Start writing the WBS (Work Breakdown Structure) of the project.
• Source Control.
• Meeting timing will be changed - hopefully to Tuesday isA.
  

For source control, I will be hosting the source code at home on my desktop PC where I have Visual SVN installed and a dynamic dns account to access it. We will be using TortoiseSVN on each client - tested and working now a7l, just finished that!

Regarding the tasks due last Sunday:

• Fouad: Done - awaiting a live test.
• Roaa: Had a killer bug in the detection code, solved a7l - tested .. Still we have some memory leaks though (undeleted pointers).
• Alaa: Tracker is done a7l! There were some surprising special cases that are handled now - awaiting two live tests: with Mahmoud's application and the hardware.
• Mahmoud: Done - awaiting a live test.

We're kinda behind schedule, and we've got lots to do for the acmASCIS Student Chapter when the next semester starts isA. We will be traveling next week isA. Working on the project will be resumed after that isA.

Back on track ..

We're done with the exams a7l & the so-called vacation has started .. It doesn't seem like anything close to the word "vacation" except that we finally could sleep with no worries that we'll have to wake up & study the next day .. lol

So, next Sunday isA is our first meeting @ ITWorx. We planned a few tasks to finish till then:

• Fouad: Get done with the bottom architecture and the camera issues and fixes ..
• Roaa: Kill all memory leaks and change the data structure for storing the blobs into a linked list ..
• Alaa: The blob tracker!!!!
• Mahmoud: Demo application for visualizing tracked-blobs in terms of (x,y) coordinates for every blob (with ID for each) ..

I have suggested a change to the way we store blobs and discussed that with Fouad, the sorted-linked-list will save us much effort in tracking and will end up with better performance isA (will discuss that in a later post) ..

I also read, again, about how many people convert from FTIR (Frustrated Total Internal Reflection) to DI (Diffused Illumination) - used in Microsoft's Surface Computing btw!

The most obvious conclusion was that FTIR provided better output (fine, clear, well-illuminated blobs) than that of the DI .. yet, it provides worse tracking of "moving" blobs; as it's really hard to move around ur fingers while pressing them hard against the acrylic sheet - with DI u don't even have to touch that hard!

DI is also much easier to "get-it-to-work", even though a practical life-long setup is usually hard enough to accomplish .. the setup of FTIR on the other hand has the only difficulty in applying the theory - a yes/no question ..

FTIR would definitely beat DI if the "tracking" story was solved out - in other words: finding a proper compliant surface, & that is still under huge research so far .. We have a several candidates for a good compliant surface; we're waiting to experiment with them .. (I started a small secret test at home, hope it works!) .. More details about that will be discussed in later posts isA ..

Tim Roth, from nuigroup, has a blog post discussing FTIR vs DI .. Check it out for more details ..

Reminder: We need to find a different time for the weekly meeting (either Tuesday or Wednesday) coz Sunday's causing a conflict with the 2nd semester's schedule!

Sample Website & ITWorx's weekly meeting

ITWorx required that we start a blog (already done; you're here as long as u're reading this), a user-group (we created one) and a website (which is currently under construction).

They have also scheduled our weekly meeting to be on Sunday, 4.00pm to 5.30pm ..
Seems suitable for the first week of vacation, as we shall start on the 27th of Jan isA. We won't make it on the second week for we'll be going on the faculty's trip isA.
There's a chance that we might need to change the time for the weekly meeting if there's a conflict with the second semester's schedule (each two of us are from a different department).

Timeout ..

We're having a break till February due to the increasing intense of the exams which will leave Touchwork at sleep for a while ..

Here are the news updates for the past, unblogged time:

• We now have the project sponsored by Microsoft and ITWorx.
• We have two supervisors from ITWorx: Essam Salah and Mihan Samy, who both seem to be so nice, helpful and interested in our work.
• Project proposal has been updated.
• Our third seminar, which was supposed to be on the 29th of Dec., was postponed to the next term .. a7l!!!!!!!
• So far, we're good according to our time-plan, which reminds me of adding the legend to the proposal as I added the timeplan schedule without one (A).

See ya after the exams .. Ciao!