Unity-TouchScripts中使用TUIO的记录和简单的代码分析

    技术2023-06-02  83

    端午三天假,刚过完端午就被老板拉过去加班去了,端午三天假加了两天班,好了不吐槽了。记录一下Unity通过TouchScript插件中TUIO协议的使用以及代码的简单分析。

    先说一下项目的大致情况,对方通过TUIO协议发送Blob格式的消息,发送的Blob消息中的面积(Area)是一个识别的重要信息,但TouchScript中返回的是Pointer类,但这个类中并没有我需要的消息。后来分析了一下代码的流向最终拿到了需要的信息。

    TuioInput.cs

    首先分析一下Tuio的输入,先看一下Unity函数:

    /// <inheritdoc /> protected override void OnEnable() { base.OnEnable(); screenWidth = Screen.width; screenHeight = Screen.height; cursorProcessor = new CursorProcessor(); cursorProcessor.CursorAdded += OnCursorAdded; cursorProcessor.CursorUpdated += OnCursorUpdated; cursorProcessor.CursorRemoved += OnCursorRemoved; blobProcessor = new BlobProcessor(); blobProcessor.BlobAdded += OnBlobAdded; blobProcessor.BlobUpdated += OnBlobUpdated; blobProcessor.BlobRemoved += OnBlobRemoved; objectProcessor = new ObjectProcessor(); objectProcessor.ObjectAdded += OnObjectAdded; objectProcessor.ObjectUpdated += OnObjectUpdated; objectProcessor.ObjectRemoved += OnObjectRemoved; connect(); } /// <inheritdoc /> protected override void OnDisable() { disconnect(); base.OnDisable(); }

    在OnEnable函数中,首先记录了一下屏幕的宽高,其次new了三个处理类,分别处理鼠标、Blob和物体的,并且分别注册了处理类的回调,当添加时、当更新时、当移除时,最后调用了connect连接函数。按下F12追踪一下connect函数;

    private void connect() { if (!Application.isPlaying) return; if (server != null) disconnect(); server = new TuioServer(TuioPort); server.Connect(); updateInputs(); }

    可以看到此处新建一个TuioServer类,并调用器自身的Connect函数,最后调用updateInputs函数。按下F12看一下TuioServer;

    namespace TUIOsharp { public class TuioServer { public TuioServer(); public TuioServer(int port); public int Port { get; } public event EventHandler<ExceptionEventArgs> ErrorOccured; public void AddDataProcessor(IDataProcessor processor); public void Connect(); public void Disconnect(); public void RemoveAllDataProcessors(); public void RemoveDataProcessor(IDataProcessor processor); } }

    这里可以看到TuioServer类中有一个添加处理器的函数AddDataProcessor和移除处理器的函数RemoveDataProcessor,这个下边会说到。我们再看一下updateInputs函数;

    private void updateInputs() { if (server == null) return; if ((supportedInputs & InputType.Cursors) != 0) server.AddDataProcessor(cursorProcessor); else server.RemoveDataProcessor(cursorProcessor); if ((supportedInputs & InputType.Blobs) != 0) server.AddDataProcessor(blobProcessor); else server.RemoveDataProcessor(blobProcessor); if ((supportedInputs & InputType.Objects) != 0) server.AddDataProcessor(objectProcessor); else server.RemoveDataProcessor(objectProcessor); }

    可以看到再updateInputs函数内部,把再OnEnable函数中新建的三个处理类添加进TuioServer中。好的,我们再看一下TuioInput注册三个处理类的回调函数(由于对方是发送Blob消息的,这里只看一下Blob的回调函数,其他类似);

    private void OnBlobAdded(object sender, TuioBlobEventArgs e) { var entity = e.Blob; lock (this) { var x = entity.X * screenWidth; var y = (1 - entity.Y) * screenHeight; var touch = internalAddObject(new Vector2(x, y)); updateBlobProperties(touch, entity); blobToInternalId.Add(entity, touch); } }

    首先看一下参数TuioBlobEventArgs;

    namespace TUIOsharp.DataProcessors { public class TuioBlobEventArgs : EventArgs { public TuioBlob Blob; public TuioBlobEventArgs(TuioBlob blob); } }

    在追踪一下TuioBlob类;

    namespace TUIOsharp.Entities { public class TuioBlob : TuioEntity { public TuioBlob(int id); public TuioBlob(int id, float x, float y, float angle, float width, float height, float area, float velocityX, float velocityY, float rotationVelocity, float acceleration, float rotationAcceleration); public float Angle { get; } public float Width { get; } public float Height { get; } public float Area { get; } public float RotationVelocity { get; } public float RotationAcceleration { get; } public void Update(float x, float y, float angle, float width, float height, float area, float velocityX, float velocityY, float rotationVelocity, float acceleration, float rotationAcceleration); } }

    好的,在这个类中可以看到有很多信息,坐标、角度、宽度、高度等一些列信息,我需要的面积也在其中,下一步就是怎么取出数据了,由于没怎么用过TUIO,也没研究过TouchScript关于这块的内容走了很多岔子,这就不提了。关于这个类的一些参考可以看一下TUIO官网的说明,链接在这:http://www.tuio.org/?specification

    接着看OnBlobAdded函数,在函数内部可以看到x值乘以了缓存的屏幕宽度、y值乘以了缓存的屏幕高度,可以断定传过来的xy是归一化后的数字(即介于0-1之间),同时y轴翻转;紧接着调用internalAddObject函数,并传参xy,看一下internalAddObject函数;

    private ObjectPointer internalAddObject(Vector2 position) { var pointer = objectPool.Get(); pointer.Position = remapCoordinates(position); pointer.Buttons |= Pointer.PointerButtonState.FirstButtonDown | Pointer.PointerButtonState.FirstButtonPressed; addPointer(pointer); pressPointer(pointer); return pointer; }

    在这里更新了一下位置,随后调用addPointer和pressPointer两个函数,这里看一下addPointer函数;

    /// <summary> /// Adds the pointer to the system. /// </summary> /// <param name="pointer">The pointer to add.</param> protected virtual void addPointer(Pointer pointer) { manager.INTERNAL_AddPointer(pointer); }

    这里的函数实际调用了父类InputSource的函数,TuioInput类继承InputSource类,函数内部又调用了manager.INTERNAL_AddPointer函数,这里的manager是TouchManagerInstance类,一个比较核心的类。看一下INTERNAL_AddPointer函数;

    internal void INTERNAL_AddPointer(Pointer pointer) { lock (pointerLock) { pointer.INTERNAL_Init(nextPointerId); pointersAdded.Add(pointer); #if TOUCHSCRIPT_DEBUG pLogger.Log(pointer, PointerEvent.IdAllocated); #endif nextPointerId++; } }

    在这里调用了Pointer类自身的INTERNAL_Init函数,这里就不看了,INTERNAL_Init函数内部更新了一下Id并 记录了一下位置,;在调用Pointer类自身的INTERNAL_Init函数后,将其添加至pointersAdded这个list中,并自动更新下一个Id,关于pointersAdded这里先不深究,等一下再说;接着看一下pressPointer函数;

    /// <summary> /// Mark the pointer as touching the surface. /// </summary> /// <param name="pointer">The pointer.</param> protected virtual void pressPointer(Pointer pointer) { if (pointer == null) return; manager.INTERNAL_PressPointer(pointer.Id); }

    和上边相同调用了manager的函数,看一下INTERNAL_PressPointer函数;

    internal void INTERNAL_PressPointer(int id) { lock (pointerLock) { Pointer pointer; if (!idToPointer.TryGetValue(id, out pointer)) { // This pointer was added this frame if (!wasPointerAddedThisFrame(id, out pointer)) { // No pointer with such id #if TOUCHSCRIPT_DEBUG if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to PRESS but no pointer with such id found."); #endif return; } } #if TOUCHSCRIPT_DEBUG if (!pointersPressed.Add(id)) if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to PRESS more than once this frame."); #else pointersPressed.Add(id); #endif } }

    在这里可以看到函数首先会从idToPointer这个字典中尝试获取Pointer,由于取反,当字典中存在时将会执行

    pointersPressed.Add(id);

    这条语句,将id添加至pointersPressed中;当字典不存在时,会调用wasPointerAddedThisFrame进行一次判断(应该是判断是否是新添加的Pointer),同样取反,true->return,false->添加至pointersPressed。wasPointerAddedThisFrame函数内部如下:

    private bool wasPointerAddedThisFrame(int id, out Pointer pointer) { pointer = null; foreach (var p in pointersAdded) { if (p.Id == id) { pointer = p; return true; } } return false; }

    通过foreach进行判断,有意思的是遍历是pointersAdded,上边分析INTERNAL_AddPointer函数时,最终的Pointer被添加的就是pointersAdded。

    好的,经过上边一串有点长的函数调用分析,终于把OnBlobAdded函数内部中的internalAddObject函数分析完毕,请滚动一下鼠标重新看一下OnBlobAdded函数,接下来要继续分析下边的执行语句;调用updateBlobProperties函数,并把一些数据添加到blobToInternalId字典中;

    private void updateBlobProperties(ObjectPointer obj, TuioBlob target) { obj.Width = target.Width; obj.Height = target.Height; obj.Angle = target.Angle; }

    updateBlobProperties函数内部更新了一些属性,如上代码所见,我们是不是可以把Area(面积)更新一下???这样我们就可以拿到自己想要的数据了,事实证明这样是可以的,我最终也是这样解决的,理论上这篇博客已经给出了开始问题的解决方案,只需要订阅TouchManager的相应事件即可,比如TouchManager.Instance.PointersAdded、TouchManager.Instance.PointersUpdated、TouchManager.Instance.PointersRemoved……在回调时把相应的Pointer转换为ObjectPointer即可拿到area(ObjectPointer时是Pointer的子类)。但最终我继续分析了其他代码,把Pointer在TouchScript内部流通给搞明白了。所以,我们继续分析;

    OnBlobUpdated函数:

    private void OnBlobUpdated(object sender, TuioBlobEventArgs e) { var entity = e.Blob; lock (this) { ObjectPointer touch; if (!blobToInternalId.TryGetValue(entity, out touch)) return; var x = entity.X * screenWidth; var y = (1 - entity.Y) * screenHeight; touch.Position = remapCoordinates(new Vector2(x, y)); updateBlobProperties(touch, entity); updatePointer(touch); } }

    在这里会首先对blobToInternalId字典尝试获取ObjectPointer(在OnBlobAdded函数最后把OnBlobAdded添加到了blobToInternalId字典中),如果获取成功,会计算位置,并通过remapCoordinates最终调用函数把位置重新映射一下(这里就补贴其他代码了,如果有兴趣请自行查看,下同),调用updateBlobProperties(上边分析过了)更新属性,最终调用updatePointer函数,updatePointer函数内部调用manager.INTERNAL_UpdatePointer函数,该函数如下:

    internal void INTERNAL_UpdatePointer(int id) { lock (pointerLock) { Pointer pointer; if (!idToPointer.TryGetValue(id, out pointer)) { // This pointer was added this frame if (!wasPointerAddedThisFrame(id, out pointer)) { // No pointer with such id #if TOUCHSCRIPT_DEBUG if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to MOVE to but no pointer with such id found."); #endif return; } } pointersUpdated.Add(id); } }

    和INTERNAL_PressPointer函数类似,只不过最后添加的是pointersUpdated而不是pointersPressed。

    OnBlobRemoved函数:

    private void OnBlobRemoved(object sender, TuioBlobEventArgs e) { var entity = e.Blob; lock (this) { ObjectPointer touch; if (!blobToInternalId.TryGetValue(entity, out touch)) return; blobToInternalId.Remove(entity); releasePointer(touch); removePointer(touch); } }

    同样的先在blobToInternalId中尝试获取ObjectPointer,如果获取成功,则从blobToInternalId移除,且调用releasePointer函数和removePointer,这两个函数最终调用了TouchManagerInstance的INTERNAL_ReleasePointer函数和INTERNAL_RemovePointer函数;其内部实现如下:

    /// <inheritdoc /> internal void INTERNAL_ReleasePointer(int id) { lock (pointerLock) { Pointer pointer; if (!idToPointer.TryGetValue(id, out pointer)) { // This pointer was added this frame if (!wasPointerAddedThisFrame(id, out pointer)) { // No pointer with such id #if TOUCHSCRIPT_DEBUG if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to END but no pointer with such id found."); #endif return; } } #if TOUCHSCRIPT_DEBUG if (!pointersReleased.Add(id)) if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to END more than once this frame."); #else pointersReleased.Add(id); #endif } } /// <inheritdoc /> internal void INTERNAL_RemovePointer(int id) { lock (pointerLock) { Pointer pointer; if (!idToPointer.TryGetValue(id, out pointer)) { // This pointer was added this frame if (!wasPointerAddedThisFrame(id, out pointer)) { // No pointer with such id #if TOUCHSCRIPT_DEBUG if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to REMOVE but no pointer with such id found."); #endif return; } } #if TOUCHSCRIPT_DEBUG if (!pointersRemoved.Add(pointer.Id)) if (DebugMode) Debug.LogWarning("TouchScript > Pointer with id [" + id + "] is requested to REMOVE more than once this frame."); #else pointersRemoved.Add(pointer.Id); #endif } }

    这两个函数内部极为相似,不同的是最后是添加的不是同一个HashSet。

    TouchManagerInstance.cs

    先看一下几个比较眼熟的字段:

    private List<Pointer> pointers = new List<Pointer>(30); private HashSet<Pointer> pressedPointers = new HashSet<Pointer>(); private Dictionary<int, Pointer> idToPointer = new Dictionary<int, Pointer>(30); // Upcoming changes private List<Pointer> pointersAdded = new List<Pointer>(10); private HashSet<int> pointersUpdated = new HashSet<int>(); private HashSet<int> pointersPressed = new HashSet<int>(); private HashSet<int> pointersReleased = new HashSet<int>(); private HashSet<int> pointersRemoved = new HashSet<int>(); private HashSet<int> pointersCancelled = new HashSet<int>();

    我们看一下它的Update函数:

    private void Update() { sendFrameStartedToPointers(); updateInputs(); updatePointers(); }

    这里只看一下updatePointers函数:

    private void updatePointers() { IsInsidePointerFrame = true; if (frameStartedInvoker != null) frameStartedInvoker.InvokeHandleExceptions(this, EventArgs.Empty); // need to copy buffers since they might get updated during execution List<Pointer> addedList = null; List<int> updatedList = null; List<int> pressedList = null; List<int> releasedList = null; List<int> removedList = null; List<int> cancelledList = null; lock (pointerLock) { if (pointersAdded.Count > 0) { addedList = pointerListPool.Get(); addedList.AddRange(pointersAdded); pointersAdded.Clear(); } if (pointersUpdated.Count > 0) { updatedList = intListPool.Get(); updatedList.AddRange(pointersUpdated); pointersUpdated.Clear(); } if (pointersPressed.Count > 0) { pressedList = intListPool.Get(); pressedList.AddRange(pointersPressed); pointersPressed.Clear(); } if (pointersReleased.Count > 0) { releasedList = intListPool.Get(); releasedList.AddRange(pointersReleased); pointersReleased.Clear(); } if (pointersRemoved.Count > 0) { removedList = intListPool.Get(); removedList.AddRange(pointersRemoved); pointersRemoved.Clear(); } if (pointersCancelled.Count > 0) { cancelledList = intListPool.Get(); cancelledList.AddRange(pointersCancelled); pointersCancelled.Clear(); } } var count = pointers.Count; for (var i = 0; i < count; i++) { pointers[i].INTERNAL_UpdatePosition(); } if (addedList != null) { updateAdded(addedList); pointerListPool.Release(addedList); } if (updatedList != null) { updateUpdated(updatedList); intListPool.Release(updatedList); } if (pressedList != null) { updatePressed(pressedList); intListPool.Release(pressedList); } if (releasedList != null) { updateReleased(releasedList); intListPool.Release(releasedList); } if (removedList != null) { updateRemoved(removedList); intListPool.Release(removedList); } if (cancelledList != null) { updateCancelled(cancelledList); intListPool.Release(cancelledList); } if (frameFinishedInvoker != null) frameFinishedInvoker.InvokeHandleExceptions(this, EventArgs.Empty); IsInsidePointerFrame = false; }

    首先看一下pointersAdded,在这里把pointersAdded装进addedList中;

    if (pointersAdded.Count > 0) { addedList = pointerListPool.Get(); addedList.AddRange(pointersAdded); pointersAdded.Clear(); }

    最后调用updateAdded函数:

    if (addedList != null) { updateAdded(addedList); pointerListPool.Release(addedList); }

    看一下updateAdded函数:

    private void updateAdded(List<Pointer> pointers) { samplerUpdateAdded.Begin(); var addedCount = pointers.Count; var list = pointerListPool.Get(); for (var i = 0; i < addedCount; i++) { var pointer = pointers[i]; list.Add(pointer); this.pointers.Add(pointer); idToPointer.Add(pointer.Id, pointer); #if TOUCHSCRIPT_DEBUG pLogger.Log(pointer, PointerEvent.Added); #endif tmpPointer = pointer; layerManager.ForEach(_layerAddPointer); tmpPointer = null; #if TOUCHSCRIPT_DEBUG if (DebugMode) addDebugFigureForPointer(pointer); #endif } if (pointersAddedInvoker != null) pointersAddedInvoker.InvokeHandleExceptions(this, PointerEventArgs.GetCachedEventArgs(list)); pointerListPool.Release(list); samplerUpdateAdded.End(); }

    可以看到最终这些Pointer被添加进idToPointer和pointers中,在函数最后几行进行了回调,完成了一次触摸点从接收到回调的一个完整流程。

    在看一下pointersUpdated:

    if (pointersUpdated.Count > 0) { updatedList = intListPool.Get(); updatedList.AddRange(pointersUpdated); pointersUpdated.Clear(); }

    被添加进updatedList中,接着往下看:

    if (updatedList != null) { updateUpdated(updatedList); intListPool.Release(updatedList); }

    最终调用了updateUpdated函数:

    private void updateUpdated(List<int> pointers) { samplerUpdateUpdated.Begin(); var updatedCount = pointers.Count; var list = pointerListPool.Get(); for (var i = 0; i < updatedCount; i++) { var id = pointers[i]; Pointer pointer; if (!idToPointer.TryGetValue(id, out pointer)) { #if TOUCHSCRIPT_DEBUG if (DebugMode) Debug.LogWarning("TouchScript > Id [" + id + "] was in UPDATED list but no pointer with such id found."); #endif continue; } list.Add(pointer); #if TOUCHSCRIPT_DEBUG pLogger.Log(pointer, PointerEvent.Updated); #endif var layer = pointer.GetPressData().Layer; if (layer != null) layer.INTERNAL_UpdatePointer(pointer); else { tmpPointer = pointer; layerManager.ForEach(_layerUpdatePointer); tmpPointer = null; } #if TOUCHSCRIPT_DEBUG if (DebugMode) addDebugFigureForPointer(pointer); #endif } if (pointersUpdatedInvoker != null) pointersUpdatedInvoker.InvokeHandleExceptions(this, PointerEventArgs.GetCachedEventArgs(list)); pointerListPool.Release(list); samplerUpdateUpdated.End(); }

    这里和updateAdded函数类似,不过它是在idToPointer中尝试获取(idToPointer承载了大部分的工作),最后回调一次PointersUpdated。

    最后分析的是pointersRemoved:

    if (pointersRemoved.Count > 0) { removedList = intListPool.Get(); removedList.AddRange(pointersRemoved); pointersRemoved.Clear(); }

    添加至removedList:

    if (removedList != null) { updateRemoved(removedList); intListPool.Release(removedList); }

    调用updateRemoved函数:

    private void updateRemoved(List<int> pointers) { samplerUpdateRemoved.Begin(); var removedCount = pointers.Count; var list = pointerListPool.Get(); for (var i = 0; i < removedCount; i++) { var id = pointers[i]; Pointer pointer; if (!idToPointer.TryGetValue(id, out pointer)) { #if TOUCHSCRIPT_DEBUG if (DebugMode) Debug.LogWarning("TouchScript > Id [" + id + "] was in REMOVED list but no pointer with such id found."); #endif continue; } idToPointer.Remove(id); this.pointers.Remove(pointer); pressedPointers.Remove(pointer); list.Add(pointer); #if TOUCHSCRIPT_DEBUG pLogger.Log(pointer, PointerEvent.Removed); #endif tmpPointer = pointer; layerManager.ForEach(_layerRemovePointer); tmpPointer = null; #if TOUCHSCRIPT_DEBUG if (DebugMode) removeDebugFigureForPointer(pointer); #endif } if (pointersRemovedInvoker != null) pointersRemovedInvoker.InvokeHandleExceptions(this, PointerEventArgs.GetCachedEventArgs(list)); removedCount = list.Count; for (var i = 0; i < removedCount; i++) { var pointer = list[i]; pointer.InputSource.INTERNAL_DiscardPointer(pointer); } pointerListPool.Release(list); samplerUpdateRemoved.End(); }

    同样在idToPointer中尝试获取,最后完成回调。

    到这里这篇博客就基本结束了,也不总结什么了,想研究的自己去看代码琢磨琢磨就清楚了。最后说一下事件的大致调用顺序

    add-press-update-released-remove

    另外Cancell没找到。

    就这样,本人水平有限,如果有错误,欢迎大佬指正,谢谢!

    Processed: 0.011, SQL: 9