Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[manowar]

Hello,

I am trying to use OpenAL with the last version of the plugin. At the moment I simply tried to integrate the 'openAL' SoundManager + AudioClip from the old example. Well I managed to compile it but it does not work. The video disappear (grey material) and it does not play the sound. (I have sometimes some kind of voice/sound comming out from my speakers). I tried to have a look at the new fmod version, and adapt the code. However OpenAL and Fmod are quite different systems and I do not know any of them. I do not have much time to learn both of them now, so if someone could help me or at least give the steps to do in orer to be able to use OpenAL.

Thanks a lot for your help

Eric

[manowar]

Hello,

Well I made a lot of progress, I have the sound working now (I do not think it works with seeking though). My problem is now how to synchronize the sound and video.Most of the time it is fine but on special events it is wrong. For example, if while the video is playing, I keep the left mouse button clicked on the window titlebar, the video freezes (which is normal) however when I release the mouse button it starts again but the sound is not synchronized anymore. I really do not know how to solve this problem. Any ideas ? Does it mean that I have to detect this mouse click event ? Then save the position/time of the audio stream, and start again from this position when the mouse is released ?

Thanks for your help

Eric

[pjcast]

Heh.. glad you made some progress on this.. I have been busy with school last couple of days (midterms and what not ).. Seeking is very experiemental.. However, when seking is done, the audio system is basically tore down, then recreated with new info.

As far as the video stopping while clicking on the window title bar.. what title bar? the CEGUI title bar? or the actual windows window title bar? I imagine that the app loses focus, but the decoding thread doesn't know and is still decoding and timing is getting out of wack... If it is ou clicking on the window's title bar, I would suggest that you catch when your app is no longer in focus and not being updated and pause the movie.. so that the thread also pauses.

Also, if you could post your changed OpenAL audio files (header and source) in a zip file, or just as source dumps here, I will work on updating them into cvs.. as I also have to make a framelistener change still...

[manowar]

Hello

Yes I am glad I have the sound 'working'. Well seeking is not important for the moment. However, I really would like to be able to synchronize the sound. Well, when I was talking about the windows title bar, it is a win32 windows. I do not use CEGUI in my project. At the moment I have not tried to replace the sound system in your application example. I will try to do that before the end of this week. (It workds fine in my app at the moment) I will post a new version as soon as it is ready. When I grab the window and move it around and release the mouse after, the sound is not synchronised anymore. I not working on it today though, i am busy on other stuff

[pjcast]

Are you using the startRendering method of Ogre?
Or are you handling the render target updating yourself (via renderOneframe or similiar)?

In either case, you have to ensure that the graphics either get repainted all the time (when window not active, or being moved/resized), or pause the clip during such an event... There is no way for the plugin to know when your app is no longer active, and so sound will continue to be fed to the sound system.. and video will not get decoded. However, this may also be a bug that needs fixing, as even if the audio gets way far ahead, the video should be able to catch up.

I do not have too much time right now to look at this, but, I will put it on my list of things to fix/lookat in the next major version. So, you can either try to just pause it during that time, or look at submitting a patch for the meantime. Sorry, just really busy on other projects and school ATM.

[Anonymous]

pjcast> Just to let you know that I was really busy on other parts of my project that I did not have the time at all to look at openal and the plugin. It is definitely on my task list because it is really a requirement for my project.
I will certainly start looking at it tomorrow, i really need to be able to synchronise the sound and video. Well thanks again for your great plugin (Too bad I cannot use fmod for this commercial project) .

[manowar]

Hello

Well I managed to look a bit more at  the plugin using openal. I have one question though concerning the update of the audio stream. How to update the audio stream and make sure that it is synchronised with the video. I guess my problems could be linked with that. For the video, I call mClip->blitFrameCheck() At the moment I have an mAudioClip->update() function called every frame too.

Code: [Select]

void AudioClip::update()
{
//Every time through - check for empty que's and fill them
int processed = 0;
alGetSourcei( stream_source, AL_BUFFERS_PROCESSED, &processed );

while(processed--)
{
//Ogre::LogManager::getSingleton().logMessage("bool AudioClip::frameStarted()****");
ALuint buffer;
unsigned int numRead = outFIFO.read( mAudioBuffer, stream_size );
if( numRead < stream_size )
{
//No Data was read at all, hopefully next time we will have some
if( numRead == 0 )
{
//Ogre::LogManager::getSingleton().logMessage("bool AudioClip::frameStarted() AudioLow");
break;
}
//Ogre::LogManager::getSingleton().logMessage("bool AudioClip::frameStarted()$$$");

//In case we ran out of data, insert audio
//XXX: Should adjust stream time for inserted blank
unsigned int temp = stream_size - numRead;
memset( mAudioBuffer + numRead, 0, temp );

alSourceUnqueueBuffers(stream_source, 1, &buffer);
alBufferData( buffer, stream_format, mAudioBuffer, stream_size, stream_rate );
alSourceQueueBuffers(stream_source, 1, &buffer);
break;
}
alSourceUnqueueBuffers(stream_source, 1, &buffer);


alBufferData( buffer, stream_format, mAudioBuffer, stream_size, stream_rate );
alSourceQueueBuffers(stream_source, 1, &buffer);
}

int state = 0;
alGetSourcei(stream_source, AL_SOURCE_STATE, &state);

if ( state != AL_PLAYING )
{
alGetSourcei(stream_source, AL_BUFFERS_PROCESSED, &processed);
alSourcePlay(stream_source);
}
}

Which is exactly the same that you used in the old version of the plugin with openal. I had a look on the new fmod version but it is quite different (use a CustomStreamCallBack). Even if now I have improved my knowledge of openal, i do not have the time to do the same with fmod. Is it the correct way to update the audio ?

I am update myself the rendering using:

Code: [Select]

bool COgre::Render(){

if(!mRoot->_fireFrameStarted())
         return false;

      mRoot->_updateAllRenderTargets();

      if(!mRoot->_fireFrameEnded())
         return false;

 return true;
}

Well thanks a lot for your help.

[pjcast]

The only difference between the OpenAL audio class and the newer FMod class is really just some extra parameters for seeking (and, perhaps better pausing). Sync is done inside the plugin with a call to getAudioTime, as long as you keep your OpenAL buffers full of data, then there should be no sync problem. For finer sync control, your audio class can adjust the time it returns if (for instance) it recieved no audio data from the plugin, and inserted blank audio into OpenAL buffer - I did not do this with either the FMod or OpenAL classes as they maintain good enough sync.

[manowar]

Hello

Well I did some new tests with your plugin, with Fmod and OpenAL. But I always got this sync problem. It is really easy to see that the lips are not synchronised with the voice. At one point I even thought it was my video, but I tried to play it with vlc and it works fine.
I played a bit more with the GEGUI example and I have the same problems too. I wanted to fix this sync problem before sending you the OpenAL version. I will try to use another video and see if I still got the problem. Thanks

Eric

[pjcast]

Just so we are on the same page... You say you are doing tests with Fmod and OpenAL; however, I am unclear as to whether you are seeing the same problem with Fmod and OpenAL? or is this just with OpenAL?

[manowar]

Well I am getting really confused, I thought I had the problem only with OpenAL. But actually I have the problem with fmod too and even in the cegui example. What I do not understand is that it seems to be a random problem. Sometimes the video is synchronised, sometimes not. I noticed that if another app is running at the same time, for example firefox, and I give it the focus, then move the mouse for a few secs, when I return to the video, it is not synchronised anymore. I will do some more tests  on monday. The best way to see the problem is to use a video with people talking and to look at their lips....


I have the cegui example working with openal now (except the colored bars), I will probably send it to you this week.

Thanks for your help.

[pjcast]

Well I am getting really confused, I thought I had the problem only with OpenAL. But actually I have the problem with fmod too and even in the cegui example. What I do not understand is that it seems to be a random problem. Sometimes the video is synchronised, sometimes not. I noticed that if another app is running at the same time, for example firefox, and I give it the focus, then move the mouse for a few secs, when I return to the video, it is not synchronised anymore. I will do some more tests  on monday. The best way to see the problem is to use a video with people talking and to look at their lips....

Ok, this does sound like a problem with the main code.. not really an FMod or OpenAL issue... As both use the Ogre timer to maintain sync. I must have missed the fact that you said this happened when switching apps.. As, I would expect that... As, Ogre's behavior on Windows is to stop rendering when the window is no longer in focus.. On Linux, the rendering never stops.. so it works good. I think you should check for the window no longer being in focus (via isActive or something) and pause the video explicity and then unpause when focus is regained. Also, make sure (if not using the startRendering() ) that you have a message pump to actually pump messages into the Ogre winproc.

I have the cegui example working with openal now (except the colored bars), I will probably send it to you this week.

Nice. Thanls

[pjcast]

To further clarify the problem... and provide another means of working around it.. The problems lays in the fact that the Ogre Timer will continue to tick even when the app is not active.. So, when you come back to the app, the audio class is reporting (via the timer) that the time is way ahead.. So the video should jump to catch up; however, the audio is actually behind the video now. Besides pausing (the best solution), the other solution is to make your audio class more aware of the actual buffer bytes fed to the sound device and adjust the time accordingly (which would also help is blank audio was inserted due to file IO falling behind or whatnot which will also lead to lag to as these errors add up - noticable in long videos).

[manowar]

Well, first of all thanks a lot for your suggestions. However I have a few comments about what your explanations:

- First of all, it does not only happen when the application lose focus. It is only an easy/quick way to desynchronise the sound and video. when I play my video in your CEGUI example using fmod, it is not always synchronised too. Even if I do not do anything at all, just watching

- The window is sill updated/rendered even if it does not have the focus. I am not using the default Ogre loop. I created myself a win32 window and update using

Code: [Select]

bool COgre::Render(){

if(!mRoot->_fireFrameStarted())
         return false;    
   
window->update();

      if(!mRoot->_fireFrameEnded())
         return false;

 return true;
}

I am surprised that i am the only one who noticed the problem.

Thanks again for your help

Eric

[pjcast]

Hmm. I have noticed sync loss only in
1) long video clips - as errors build up over time - that should be accounted for by keeping track of how many bytes have been fed into the sound buffer (either fmod or OpenAL)
2) When seeking is enabled - before an initial seek is done, after the first seek it regains sync
3) certain video clips I have transcoded from whatever format (mpeg, divx, etc) to theora.. Have not narrowed this down yet... Though, it happens all the time, not just occasionally
4) If the quaility (as set when using 'ffmpeg2theora -v #' command) is too high - a bug in my code that cannot deal with the high bitrate I guess

Maybe one of those things is happening to you? A very rare chance.. I have heard that sometimes (though, I do not know how accurate this info is) QueryPerformanceTimer is wrong/jumpy on some hardware...- as this is the default Ogre timer created with Ogre Platform Manager. A hack, that may not even help you at all, is to bypass the platform manager createTimer call in the OpenAL/Fmod audio driver creation.. and instead create the stock (TimeGetTime?) Win32 timer that is inside the Win32Platform Manager... I directly created that timer before (and for me both worked good) when testing a Timer::pause method patch I added to Ogre (which never got approved because the Ogre linux/Mac maintainer could not be bothered to add that OS implementation). :/