27th February was spent over looking the progress of the prototype and what problems there are. The top issue was still trying to edit anything within the tbeta actionscript code because as it turns out, actionscript 3 is almost a different architecture to actionscript 2. This was a big issue for the group because the knowledge only went as far as actionscript 2. So a solution round this was to send data out of the as3 file and then receive it via the as2 file. This would require next to no knowledge of as3 because the group would ultimately be working in as2.

Research was carried out on all types of ways to do this but to no avail. Nothing worked with the groups as3 script initially and this issue continued.

Development has stopped in regards to the concept and user experiences due to the briefs requirement to have a working prototype for the project. Focus was now to be spent on completing the presentation boards, prototype hardware and completing scripting.

28th February was spent building the prototype framework and although took a long time, was very successful and really looks professional. The group was happy with the results and was later packed up ready for transport into Ravensbourne.

The prototype framework was taken into Ravensbourne on the 3rd March and was assembled relatively quickly compared to Saturdays efforts. The same day, the perspex for the project arrived and was a real sense of achievement as (hardware wise) everything was coming together.

As this only took up the morning to complete, the rest of the afternoon was spent writing out what content the group wanted in the presentation boards, how many and yet another template was created.

Progress on the scripting was really slow and the group was not getting anywhere fast. So the issue was posted on a forum for additional help as there is not anyone in Ravensbourne that has a good understanding of flash actionscript.

Finally the group got a reply from the user ‘Parker’ on the 4th March that summed up the reason why we were having issues with as3. The group simply had to import a class that will enable the localConnection variable.

http://nuigroup.com/forums/viewthread/4634/

This really pushed the scripting issues forward and a lot of progress was made shortly after with both actionscript files talking to each other. This also brought up so many other issues that the project looks very unlikely to have anything fully working for the 11th March. With these massive script issues, presentation boards and the groups personal project logs to make, it looks like this last week is going to be very difficult.

Today was the formative formative. Meaning that there is a kind of dress-rehearsal before the actual formative assessment with an external examiner next wednesday.

A forum reply on the NUI Group forum helped with a coding issue but that is far from resolved.

A number of things have happened this week including the perspex arriving and being installed into the frame and the frame being secured to it’s feet.

The perspex screen was plastered with tracing paper and celotaped to prevent movement or ripping between the sheets The setup was tested in place and seemed to be ok however the webcam was inadequate due to reflections from the perspex

The webcam could not be hacked to only show IR as the existing filter is inside the lens itself rather than at the end of it. So a new webcam was ordered that has been confirmed to do this, when it arrives the process will restart.

Thursday some wood was purchased from the wood-a-torium (B&Q) as well as some fixings. The decision was made to use metal brackets to attach the feet to the frame rather than braces because it was easier and would look better.

The wood for the vertical supports were marked at suitable heights and four lengths were cut equally. The same was done for the feet.

The location of the screen was market and holes were drilled above and below it. This allows the perspex sheet to be pinned between the wooden layers and not have to directly drill it. Hopfeully this means there is more chance of getting reimbursement from the college as it can be re-used. Bolts were put through the holes and washers then nuts were place on the other side and tightened.

The metal ‘L’ brackets were screwed in at the bottom of each piece of wood with wood screws. The feet were left off for transportation but just need to be screwed into. The position of where the vertical supports would go has been marked to make things easier.

The perspex should arrive on Monday and they it will be time to test fit the whole setup. Tracing paper will be added as the projection and diffusion surface and a projector will need to be requisitioned from somewhere in order to test the tracking system. All that remains is the retro fitting of the webcam with an IR filter, Something that could be either very easy or very difficult; it appears to be leaning towards the latter.

The IR LED lamp that was purchase decided not to work when plugged in yesterday but after some quick correspondance with the seller it was discovered that a light sensor was on the back so it would only come on in the dark. A level of automation for IR CCTV cameras. This problem was quickly soved with some blu-tack.

This week will be spent purchasing and (hopefully) constructing the projection surface. While the surface itself is just a perspex sheet with tracing paper as the diffuser, this needs to be held up by something. So a frame support needs to be built.

Materials:

• 4 x 1950mm vertical supports
• 2 x 650mm feet
• 2 x 300mm angled braces
• Wing nuts, washers & bolts

The frame needs to be this size to be this size to fit within double doors between two rooms. This allows environmental control over the front and rear of the touch screen. So the lights can be turned off and the blinds shut to control ambient light in the back of the touch surface to allow a better spread of IR light from the IR LED lamp.

Using wing nuts and bolts allows the whole system to be easily dismantled and transported.

The structure will follow the principles in the instructable found here [http://www.instructables.com/] but the sizes will be slightly different.

The next step when the contruction is complete is to hack a webcam to only allow infrared light through. There will be more pictures in that post to show the detail and hopfeully how it works afterwards. Otherwise a new webcam needs to be purchased.

Work has really been picking up so these entries may be more common from now on. Yesterday began with Flash Actionscript to get the prototype moving along in terms of not having multi touch all linked up and the rest of it, but just simply mouse clicking objects to get feedback. This resulted in being able to draw an object, visually present that being an object and then giving that a simple menu. The plan is to get everything working via mouse input and then import than class’s required for the touch inputs in there as well.

Here is a list of problems that raised:

• Once a shape was drawn, having that held into a movieclip.
• Accessing objects from an Array to be manipulated.
• Having previous objects stay on screen, not disappear when another is created.

Today involved talking to individuals that do not know anything about this project and proposed current navigation concepts. Through this new gesture navigation was brought up and was well received from the participants. Originally the navigation was to create a shape by touch and that object, be as dynamic as possible, would be the outline of whatever device chosen by the user. The problem that raised through development was creating these shapes was difficult, even to link a circle up proved awkward so the alternative to this was the user drawing a quick line, which dependant on its max height, length and smallest height, length displayed the outline creating a box. This would then give the option of what type of shape the user wanted this to be. Instantly proved more intuitive and easier for the user to grasp.

Yesterdays issues where looked at again today but still no luck. So it was decided to do a different aspect of the project and start creating the framework for the navigation and what would ultimately be involved. This will be developed over time and user research will undoubtedly change the layout significantly.

14th February was spent on sketching presentation board templates and finalizing the concept navigation before user testing is introduced in Illustrator.

On the 17th the group worked on improving actionscript code with some success resulting in a multi touch painting program but have looked at the progression of the project and prioritized aspects including what’s working and what is not. Depending on Judy’s feedback tomorrow, coding will be dropped for a video representation in replacement for presentation quality. Also including sketching storyboards for the video presentation and ideas which would portray the project as best as possible.

The 18th started with talking to Judy and was sent to see James Uren on the possibility of solving problems the group have recently been trying to fix. This resulted in answering one question but still leaves a lot to be desired in the final stages as the group believes at this point, the quality would be significantly better creating a video based on what is happening instead of prototyping. So in conjunction with yesterday’s decision, the prototype progress will stop. But a prototype is required, so although not particularly interesting, the current prototype will be shown in the final presentation along side high quality presentation boards and video. Stop motion, 3D or just normal video has yet to be decided on.

That evening the group researched potential programs which would best demonstrate the project such as 3D Studio Max, After Effects, Photoshop and Illustrator. A few shots were taken at 3D Studio Max and Photoshop but was decided to concentrate in Illustrator for the boards because of the vectors flexibility.

Today the survey was sent out and currently has over 140 replies which is excellent news and the results are really interesting. Also research was conducted on inspiration for the presentations style and feel. Coming across the lovely vector made by 3LgoRdo, “My Beloved Bella…” was very influencing and including other artwork across the net, this image really ticked the box. http://3lgordo.deviantart.com/art/My-Beloved-Bella-113359509

This image decided the feel and style which resulted in this, a representation of the project concept. This is currently in progress and will carry the next couple of boards in the right direction.

This week has been difficult to make progress as the Flash application development has been going badly. The lack of basic actionscript knowledge has been a hinderence to say the least. The tracking system works poerfectly and the data is being sent to Flash perfectly but it is when the data has arrived in Flash that the probems arise.

So as always, start at the very basic and work your way up. In this instance it would be a simple drawing application in Flash, forgetting the touch screen aspect of it. This was complete with relative ease but mainly due to follwing a very good tutorial and not really understanding the principles behind it.

Then the tutorial code was pasted into an existing multi-touch application in an attempt to draw using touch, this resulted in errors being spat out as soon as the application was run. The problem was because the two were incompatible and written with very different requirements in mind. A few hours were spent trying to ‘trace’ the x and y coordinates from the tracking information and with help it was understood where the information was being stored and so there was now access to the data being sent from the tracking software.

This is of little help as trying to do anything to do with gestures has proved fruitless. The aim has been dramatically reduced from the original prototype of drawing a shape and having that shape take on a specific role (TV, speaker, light) to simply bringing up a menu when a gesture is made.

There are just about three weeks left of this project so the decision has been made to focus more on the presentation of the concept rather than building of the prototype. The prototype will still be built but as a multi-touch paint application to show the ability to build the technology allowing for multi-touch rather than the application.

To demonstrate the application there will be a series of slides/animations that can be ‘clicked’ on and show on a real scale what would happen when the wall is touched in various ways. This combined with a looping video and four A3 board should provide enough information to understand the concept.

This week consisted of sketching interface and gesture concepts for the interactive project which came out really nicely and many creative ideas were popping out. These ranged from circular, futuristic and floating menu designs due to the nature of the project. People are not robots, thus can never draw perfect square boxes or circles consistently, so the way the menu appears would be a floating type action and very dynamic to how and where the user finishes creating their shapes. For instance, if the user created a box starting from the bottom right and went round in a clockwise fashion, the menu would appear on the right as that is where the user’s finger was at the end.

Also some ongoing secondary research has progressed in terms of getting influences from already established prototypes on the net. Even simply looking at illustrations for colours, styles and lighting are so inspiring and the group really wants to get a unique feel and intuitive sense for the project. Here are two youtube videos and illustrations to show.

http://www.youtube.com/watch?v=9WM8wL24USc&feature=channel
http://www.youtube.com/watch?v=RetJ_W6s-V8&feature=channel_page

http://mariyumi.deviantart.com/art/Lute-106890415
http://yanabondareva.deviantart.com/art/100-102658039
http://luminatii.deviantart.com/art/Organic-REmix-66228502
http://jomino.deviantart.com/art/io-97517210

These and many others gave ideas on how the concept would look and feel from a digital perspective and the way finger inputs would link up to create a centre area as shown in the first youtube video. It had a potently good place to create a menu if the user does not desirer making shapes for their media.

Additionally a number of questions were written down for a survey which will be setup at http://www.surveymonkeys.com. This site was used by many students last term and served them all very well with a number of options for questions and statistics after. Although they are rough, these give an idea on what areas the group are looking into asking. These will be pushed around, removed and improved on what’s relevant to the project.

• Is your wall cluttered/messy and full of images? Because of devices?
• What type of stuff is on your wall?
• Do you think 0.3mm televisions and… is becoming “silly”? Pointless
• How many screens/lights/media in your room?
• Is thinner better?
• When interaction with objects, what do you find easier to navigate with? Colour, sound, touch or feeling?
• Do you fill up walls? Is it colourful/vibrate?
• What type of television do you have?
• Do you care about all this high definition?
• Do wires and linking up devices annoy you?
• Do you keep things tidy?
• Do you currently see interactive walls as a gimmick or potentially something greater could come out of this… way of presenting information?
• Do you like these new touch-screen phones? (iPhone, LG)
• Would you say your quite technical as in know the ins and outs of your “setup”. Television, speakers, wire setup etc.
• Do you prefer a “don’t care as long as it works” attitude towards technology or do you like the option to play with settings?
• Do you find all these individual devices scattered across your home an inconvenience? Would you prefer that device to follow you instead?

The trouble with trying to make a multi-touch application is that there are so many different things to consider. Hardware wise there are many ways to create a touch screen including LLC, FTIR and DI. These use various aspects of reflection and camera tracking to find out where the surface has been touched. Details of the various techniques can be found here [http://nuigroup.com/forums/viewthread/1982/]

The large scale demonstration will consist of a large perspex sheet, a diffuser (such as tracing paper), a projector, a camera with an IR filter and two IR lights. All the equipment will be behind the screen and the IR lights shine onto the screen as well as the projector. The diffuser allows the IR light to diffuse through it and when a person touches the surface, the camera tracks the reflection of IR light and passes that through various filters to allow for a clear blob which is then given an ID and passed into a third party application. The tracking software used is here [http://tbeta.nuigroup.com/]

There are a few software options for building a multi-touch application and a variety of languages to program in (C++, Java, Processing) but the concentration is on finding a framework that will make things easier. Sparsh UI is supposedly good although this has yet to work. There is a plugin for TouchLib (but TBeta is better and is what is being used for tracking) which sends TUIO data (the same as TBeta) but there is a problem preventing connection of the two at the moment. VVVV is a multipurpose tooklit that is similar in style to MaxMSP but there are some connection issues there too.

Flash seems to be the best option as it has been connected but there is so much involved in all the tutorial files that there has been some failure in writing the code from scratch.

The decision has been made to start working on the application within flash but without the inputs from the MT. This means that work can still progress as well as research into how to sync the information being transmitted properly.

For the final presentation the setup needs to be a scaled down, simplified version of the concept however it does need to convey some of the scale and scope of the concept. To this end the size of the projection screen needs to be about 2000mm x 1000m onto which will be placed tracing paper or a this translucent membrane of material in order to hold the projection (buying a holo-film projection screen is not cost effective for the demonstration).

Emails have been sent to various plastics manufaturers and distributers to see what they could offer in terms of freebies and discounts. One email has been received back today from http://www.theplasticshop.co.uk saying that they were unable to offer the project any materials for free but would be able to offer 20% discount.

2000mm x 1500mm x 3mm Clear EXTRUDED Perspex @ £51.70-20% each
Carriage and Packaging @ £18.95 within England and Wales.
The lead time is 3-4 days.
The above prices are excluding VAT.

When or if the other companies reply, the course budget may be able to contribute to the purchase as long as the materials can be reused for the course. Considering the course is prodominantly screen based and the penetration of touch-screen devices in the consumer market, it is difficult to see why funding would NOT be given.