3D label displays abnormally.

[zhouhongbinboy]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for main branch).

Describe the issue

13D Label (add_3d_label) Display Issue:
When multiple scenes (SceneWidget) share a single window, the 3D labels only exhibit abnormal behavior in specific scenes (e.g., Scene 3).
In these abnormal scenes, the 3D labels are invisible within the normal zoom range but appear at the correct coordinate points when the camera is zoomed in extremely close.
During zooming in or out, the label positions exhibit a "drifting away" phenomenon, deviating from the coordinate points.

2Issue Related to Scene Layout:
The abnormal behavior occurs when the top border of the problematic scene is positioned below the upper edge of the window.
Moving the top border of a normally functioning scene downward causes the same issue to appear, and the farther the scene's top border is from the window's upper edge, the more severe the issue becomes.
Under the same code and configuration, other scenes do not exhibit similar problems.
TriAligner.txt

Steps to reproduce the bug

import open3d as o3d
import open3d.visualization.gui as gui
import open3d.visualization.rendering as rendering
# import plotly.express as px
import numpy as np
import os.path
import threading
import copy
colors = [
    [0.98, 0.02, 0.04],
    [0.01, 0.96, 0.05],
    [0.03, 0.05, 0.94],
    [0.91, 0.94, 0.01],
    [0.88, 0.03, 0.89],
    [0.07, 0.88, 0.91],
    [0.91, 0.09, 0.88],
    [0.91, 0.88, 0.93],
    [0.15, 0.15, 0.13],
    [0.68, 0.65, 0.95]
]

# 获取当前文件的绝对路径
basedir = os.path.dirname(os.path.realpath(__file__))
# 定义一个类来存储每个场景的特定数据
class SceneData:
    def __init__(self):
        self._picked_indicates = []  # 存储被拾取点的索引
        self._picked_points = []     # 存储被拾取点的坐标
        self._pick_num = 0           # 当前拾取点的计数
        self._label3d_list = []      # 存储3D标签

class App:
    # 定义菜单项的ID
    MENU_OPEN = 1
    MENU_SHOW = 5
    MENU_QUIT = 20
    MENU_ABOUT = 21

    show = True

    # 初始化一个空列表来存储线段
    lines = []

    def __init__(self):
        # 初始化 GUI 组件
        self.initialize_gui()
        print('done')

    def initialize_gui(self):
        gui.Application.instance.initialize()
        self.window = gui.Application.instance.create_window("Collector", 1920, 1080)  # 创建窗口实例
        global w  # 使用全局变量 w
        global bottomSpace
        bottomSpace = 120  # 窗口底部预留的空间高度，用来放置场景下方的控件
        w = self.window
        em = w.theme.font_size  # 使用字体大小作为参考，控件大小随之调整

        self.pageIndex = 1

        self.cameraToOriginDistance = 100  # 相机到原点的距离
        self.fov = 20  # 视野角度
        self.extraDistance = 500  # 相机到原点的额外距离

        r = self.window.content_rect  # 获取窗口的大小

        # 创建3个不同场景，用于容纳3个不同3d图形
        self.scene_datas = []  # 用于存储不同场景中的数据
        self._scene1 = gui.SceneWidget()  #合并后物体场景
        # 在添加坐标系后，添加测试标签
        self._scene1.scene = rendering.Open3DScene(w.renderer)
        self._scene2 = gui.SceneWidget()  #待连接物体1的场景
        self._scene2.scene = rendering.Open3DScene(w.renderer)
        self.scene_datas.append(SceneData())
        self._scene2.set_on_mouse(self._make_on_mouse_widget3d(1))  # 为场景2设置独立的鼠标事件处理
        self._scene3 = gui.SceneWidget()  #待连接物体2的场景
        self._scene3.scene = rendering.Open3DScene(w.renderer)
        self.scene_datas.append(SceneData())
        self._scene3.set_on_mouse(self._make_on_mouse_widget3d(2))  # 为场景3设置独立的鼠标事件处理
        print("Scene1 renderer settings:", self._scene1.scene)
        print("Scene1 renderer settings:", self._scene2.scene)
        print("Scene3 renderer settings:", self._scene3.scene)

        # 信息标签，用于显示所抓取的点的坐标
        self._info = gui.Label("")  # 创建空标签
        self._info.visible = False  # 初始不可见
        # 设置布局之间的距离，及布局内到四条边的留白
        self.layoutDown = gui.Vert(0, gui.Margins(0.35 * em, 0.35 * em, 0.35 * em, 0.35 * em))
        self.p1layoutRight = gui.Vert(0, gui.Margins(0.5 * em, 0.5 * em, 0.5 * em, 0.5 * em))
        self.p2layoutRight = gui.Vert(0, gui.Margins(0.5 * em, 0.5 * em, 0.5 * em, 0.5 * em))
        self.p3layoutRight = gui.Vert(0, gui.Margins(0.5 * em, 0.5 * em, 0.5 * em, 0.5 * em))
        self.p4layoutRight = gui.Vert(0, gui.Margins(0.5 * em, 0.5 * em, 0.5 * em, 0.5 * em))
        self.p5layoutRight = gui.Vert(0, gui.Margins(0.5 * em, 0.5 * em, 0.5 * em, 0.5 * em))
        self.p6layoutRight = gui.Vert(0, gui.Margins(0.5 * em, 0.5 * em, 0.5 * em, 0.5 * em))

        # 添加布局到窗口
        w.add_child(self._info)
        w.add_child(self._scene1)
        w.add_child(self._scene2)
        w.add_child(self._scene3)
        w.add_child(self.layoutDown)
        w.add_child(self.p1layoutRight)
        w.add_child(self.p2layoutRight)
        w.add_child(self.p3layoutRight)
        w.add_child(self.p4layoutRight)
        w.add_child(self.p5layoutRight)
        w.add_child(self.p6layoutRight)

        #_________________________layoutDown布局的子元素:
        # 文本编辑器占据尽可能多的空间
        fileedit_layout = gui.Horiz()
        fileedit_layout.add_child(gui.Label("File Path"))

        # 创建文件选择控件，一部分是文本编辑器用于显示文件路径，点击按钮可以选择文件
        self._fileedit = gui.TextEdit()
        fileedit_layout.add_child(self._fileedit)

        filedlgbutton = gui.Button("...")
        filedlgbutton.horizontal_padding_em = 0.5
        filedlgbutton.vertical_padding_em = 0
        filedlgbutton.set_on_clicked(self._menu_open)  # 调用与菜单打开相同的函数

        fileedit_layout.add_fixed(0.25 * em)
        fileedit_layout.add_child(filedlgbutton)
        # 添加到顶层垂直布局
        self.layoutDown.add_child(fileedit_layout)
        self.layoutDown.add_fixed(0.4 * em)

        self.layoutDown2 = gui.Horiz()  # layoutDown的第二行

        # 状态栏
        self.stateLy = gui.Horiz(0, gui.Margins(0, 0, 0, 0))
        self.stateLy.add_child(gui.Label("State:"))
        self.stateTxt = gui.TextEdit()
        self.stateTxt.placeholder_text = "idle"
        self.stateLy.add_child(self.stateTxt)
        self.layoutDown2.add_child(self.stateLy)
        
        self.layoutDown2.add_fixed(5 * em)
        self.layoutDown2.add_child(gui.Label("Device Ports:"))

        dp = gui.ListView()
        dp.set_items(["Ground", "Trees", "Buildings", "Cars", "People", "Cats", "Ground", "Trees", "Buildings", "Cars", "People", "Cats", "Ground", "Trees", "Buildings", "Cars", "People", "Cats"])
        self.layoutDown2.add_child(dp)
        self.layoutDown.add_child(self.layoutDown2)


        #_________________________p1layoutRight布局的子元素:
        self.p1layoutRight.add_fixed(1.4 * em)
        self.p1Label1 = gui.Label("Make sure markers show up and can be moved by users.")
        self.p1Button2 = gui.Button("Next Step")
        self.p1Button2.set_on_clicked(self.button2Click)
        self.p1layoutRight.add_child(self.p1Label1)
        self.p1layoutRight.add_fixed(0.4 * em)
        self.p1layoutRight.add_child(self.p1Button2)

        #__________________________p2layoutRight布局的子元素:
        self.p2layoutRight.add_fixed(1.4 * em)
        self.p2Label1 = gui.Label("The upper and lower teeth naturally bite together perfectly, then start recording.")
        self.p2Button1 = gui.Button("Previous Step")
        self.p2Button1.set_on_clicked(self.button1Click)
        self.p2Button2 = gui.Button("Record")
        self.p2Button2.set_on_clicked(self.button2Click)
        self.p2layoutRight.add_child(self.p2Label1)
        self.p2layoutRight.add_fixed(0.4 * em)
        self.p2layoutRight.add_child(self.p2Button1)
        self.p2layoutRight.add_fixed(0.4 * em)
        self.p2layoutRight.add_child(self.p2Button2)

        #__________________________p3layoutRight布局的子元素:
        self.p3layoutRight.add_fixed(1.4 * em)
        self.p3Label1 = gui.Label("Move forward and backward.")
        self.p3Button1 = gui.Button("Previous Step")
        self.p3Button1.set_on_clicked(self.button1Click)
        self.p3Button2 = gui.Button("Record")
        self.p3Button2.set_on_clicked(self.button2Click)
        self.p3layoutRight.add_child(self.p3Label1)
        self.p3layoutRight.add_fixed(0.4 * em)
        self.p3layoutRight.add_child(self.p3Button1)
        self.p3layoutRight.add_fixed(0.4 * em)
        self.p3layoutRight.add_child(self.p3Button2)

        #__________________________p4layoutRight布局的子元素:
        self.p4layoutRight.add_fixed(1.4 * em)
        self.p4Label1 = gui.Label("Move to left and right.")
        self.p4Button1 = gui.Button("Previous Step")
        self.p4Button1.set_on_clicked(self.button1Click)
        self.p4Button2 = gui.Button("Record")
        self.p4Button2.set_on_clicked(self.button2Click)
        self.p4layoutRight.add_child(self.p4Label1)
        self.p4layoutRight.add_fixed(0.4 * em)
        self.p4layoutRight.add_child(self.p4Button1)
        self.p4layoutRight.add_fixed(0.4 * em)
        self.p4layoutRight.add_child(self.p4Button2)

        #__________________________p5layoutRight布局的子元素:
        self.p5layoutRight.add_fixed(1.4 * em)
        self.p5Label1 = gui.Label("Move upward and downward.")
        self.p5Button1 = gui.Button("Previous Step")
        self.p5Button1.set_on_clicked(self.button1Click)
        self.p5Button2 = gui.Button("Record")
        self.p5Button2.set_on_clicked(self.button2Click)
        self.p5layoutRight.add_child(self.p5Label1)
        self.p5layoutRight.add_fixed(1 * em)
        self.p5layoutRight.add_child(self.p5Button1)
        self.p5layoutRight.add_fixed(0.4 * em)
        self.p5layoutRight.add_child(self.p5Button2)

        #__________________________p6layoutRight布局的子元素:
        self.p6layoutRight.add_fixed(1.4 * em)
        self.p6Label1 = gui.Label("Finish.")
        self.p6Button1 = gui.Button("Previous Step")
        self.p6Button1.set_on_clicked(self.button1Click)
        self.p6Button2 = gui.Button("Replay")
        self.p6Button2.set_on_clicked(self.replayButtonClick)
        self.p6layoutRight.add_child(self.p6Label1)
        self.p6layoutRight.add_fixed(0.4 * em)
        self.p6layoutRight.add_child(self.p6Button1)
        self.p6layoutRight.add_fixed(0.4 * em)
        self.p6layoutRight.add_child(self.p6Button2)

        # 设置布局回调函数
        w.set_on_layout(self._on_layout)

        # 如果没有菜单栏，则创建一个菜单栏
        if gui.Application.instance.menubar is None:
            file_menu = gui.Menu()
            file_menu.add_item("Open", App.MENU_OPEN)
            file_menu.add_separator()
            file_menu.add_item("Quit", App.MENU_QUIT)

            show_menu = gui.Menu()
            show_menu.add_item("Show Upper Jaw", App.MENU_SHOW)
            show_menu.add_item("Show Lower Jaw ", App.MENU_SHOW)
            show_menu.set_checked(App.MENU_SHOW, True)
            show_menu.set_checked(App.MENU_SHOW, True)

            help_menu = gui.Menu()
            help_menu.add_item("About", App.MENU_ABOUT)
            help_menu.set_enabled(App.MENU_ABOUT, False)

            menu = gui.Menu()
            menu.add_menu("File", file_menu)
            menu.add_menu("Show", show_menu)
            menu.add_menu("Help", help_menu)

            gui.Application.instance.menubar = menu

            w.set_on_menu_item_activated(App.MENU_OPEN, self._menu_open)
            w.set_on_menu_item_activated(App.MENU_QUIT, self._menu_quit)
            w.set_on_menu_item_activated(App.MENU_SHOW, self._menu_show)

        # 加载初始文件
        # self.loadUp('zhangsan20231229001\\upScan.stl')

    # 创建鼠标事件回调函数的工厂方法
    def _make_on_mouse_widget3d(self, idx):
        def _on_mouse_widget3d(event):
            # 获取对应场景的数据
            scene_data = self.scene_datas[idx-1]
            scene_widget = getattr(self,f"_scene{idx+1}")
            # 如果按下左键并按住Ctrl键
            if event.type == gui.MouseEvent.Type.BUTTON_DOWN and event.is_button_down(gui.MouseButton.LEFT) and event.is_modifier_down(gui.KeyModifier.CTRL):

                def depth_callback(depth_image):
                    # 获取鼠标点击位置的深度值
                    x = event.x - scene_widget.frame.x  # 鼠标x坐标相对于场景窗口的偏移
                    y = event.y - scene_widget.frame.y  # 鼠标y坐标相对于场景窗口的偏移
                    depth = np.asarray(depth_image)[y, x]  # 获取深度图中对应点的深度值

                    if depth == 1.0:
                        # 如果深度为1.0，表示未命中任何几何体
                        text = ""
                    else:
                        # 将屏幕坐标转换为世界坐标
                        world = scene_widget.scene.camera.unproject(
                            x, y, depth, scene_widget.frame.width, scene_widget.frame.height
                        )
                        text = "({:.3f}, {:.3f}, {:.3f})".format(world[0], world[1], world[2])  # 格式化显示坐标

                        idx_point = self._cacl_prefer_indicate(world)  # 计算最接近的点索引
                        true_point = np.asarray(self.pcd.points)[idx_point]  # 获取点云中的真实点

                        scene_data._pick_num += 1  # 更新拾取点计数
                        scene_data._picked_indicates.append(idx_point)  # 保存索引
                        scene_data._picked_points.append(true_point)  # 保存点坐标

                        print(f"在场景 {idx+1} 中拾取点 #{idx_point}，坐标为 ({true_point[0]}, {true_point[1]}, {true_point[2]})")  # 输出拾取信息

                    def draw_point():
                        # 更新信息标签
                        self._info.text = text
                        self._info.visible = (text != "")
                        self.window.set_needs_layout()  # 请求布局更新

                        if depth != 1.0:
                            # 在拾取点位置添加3D标签
                            r = self.window.content_rect  # 获取窗口的大小
                            em = w.theme.font_size
                            Height = r.height - 3.75 * em
                            print(true_point)
                            # true_point[2] = true_point[2] + Height/2
                            label3d = scene_widget.add_3d_label(true_point, "#" + str(scene_data._pick_num))
                            scene_data._label3d_list.append(label3d)
                            print(len(scene_data._label3d_list))
                            # 在拾取点位置绘制红色小球
                            sphere = o3d.geometry.TriangleMesh.create_sphere(2.5)
                            sphere.paint_uniform_color(colors[scene_data._pick_num-1])  # 设置颜色
                            sphere.translate(true_point)  # 移动到拾取点位置
                            material = rendering.MaterialRecord()
                            material.shader = 'defaultUnlit'  # 无光照材质
                            scene_widget.scene.add_geometry("sphere" + str(scene_data._pick_num), sphere, material)
                            scene_widget.force_redraw()  # 强制重绘

                    gui.Application.instance.post_to_main_thread(self.window, draw_point)  # 将绘制任务加入主线程

                scene_widget.scene.scene.render_to_depth_image(depth_callback)  # 渲染深度图并调用回调函数
                return gui.Widget.EventCallbackResult.HANDLED  # 表示事件已处理

            # 如果按下右键并按住Ctrl键（撤销）
            elif event.type == gui.MouseEvent.Type.BUTTON_DOWN and event.is_button_down(gui.MouseButton.RIGHT) and event.is_modifier_down(gui.KeyModifier.CTRL):
                if scene_data._pick_num > 0:
                    # 撤销最后一次拾取
                    idx_point = scene_data._picked_indicates.pop()
                    point = scene_data._picked_points.pop()

                    print(f"在场景 {idx+1} 中撤销拾取点 #{idx_point}，坐标为 ({point[0]}, {point[1]}, {point[2]})")  # 输出撤销信息

                    scene_widget.scene.remove_geometry('sphere' + str(scene_data._pick_num))  # 移除红色球体
                    scene_data._pick_num -= 1  # 减少计数
                    scene_widget.remove_3d_label(scene_data._label3d_list.pop())  # 移除3D标签
                    scene_widget.force_redraw()  # 强制重绘
                else:
                    print(f"场景 {idx+1} 中没有可撤销的点！")  # 无拾取点可撤销
                return gui.Widget.EventCallbackResult.HANDLED  # 表示事件已处理

            return gui.Widget.EventCallbackResult.IGNORED  # 未处理的事件
        return _on_mouse_widget3d

    # 计算最近点的索引
    def _cacl_prefer_indicate(self, point):
        pcd = copy.deepcopy(self.pcd)  # 深拷贝点云
        pcd.points.append(np.asarray(point))  # 将拾取点添加到点云中

        pcd_tree = o3d.geometry.KDTreeFlann(pcd)  # 构建KD树
        [k, idx, _] = pcd_tree.search_knn_vector_3d(pcd.points[-1], 2)  # 查找最近邻
        return idx[-1]  # 返回最近点索引


    def _on_layout(self, layout_context):
        """
        窗体首次创建，及每次调整窗口大小时均会调用该函数
        """
        # 布局所有的元素
        print(layout_context)
        r = self.window.content_rect  # 获取窗口的大小
        em = w.theme.font_size  # 使用字体大小作为参考

        global bottomSpace
        bottomSpace = 3.75 * em  # 底部布局的高度
        rpannel_height = r.get_bottom() - r.y
        laytoutRightWidth = 12.41 * em
        layoutScenesWidth = r.width - laytoutRightWidth
        layoutScenesHeight = r.height - bottomSpace
        dpannel_width = layoutScenesWidth  # 参考场景的宽度
        sceneWidth = layoutScenesWidth
        sceneHeight = layoutScenesHeight
        _fileeditHeight = 2 * em
        stateLyWidth = 6 * em
        # 布局场景和其他控件
        self._scene1.frame = gui.Rect(r.x+sceneWidth/2+1, r.y, sceneWidth/2-1, sceneHeight)
        self._scene2.frame = gui.Rect(r.x, r.y, sceneWidth/2-1, sceneHeight/2-1)
        self._scene3.frame = gui.Rect(r.x, r.y+sceneHeight/2+1, sceneWidth/2-1, sceneHeight/2-1)
        # self.line1.frame = gui.Rect(r.x+sceneWidth/2-1,r.y,2,sceneHeight)
        # self.line2.frame = gui.Rect(r.x,r.y+ sceneHeight/2-1,sceneWidth/2+1,2)
        self.layoutDown.frame = gui.Rect(r.x + 2, r.y + sceneHeight + 5, dpannel_width, bottomSpace)
        self._fileedit.frame = gui.Rect(r.x + 2, r.y + sceneHeight + 5, dpannel_width, _fileeditHeight)
        self.layoutDown2.frame = gui.Rect(r.x + 2, r.y + sceneHeight + 5 + _fileeditHeight, dpannel_width, _fileeditHeight)
        self.stateLy.frame = gui.Rect(r.x + 2, r.y + sceneHeight + 5 + _fileeditHeight, stateLyWidth, _fileeditHeight)
        self.p1layoutRight.frame = gui.Rect(r.x + layoutScenesWidth + 2, r.y, laytoutRightWidth, rpannel_height)
        self.p2layoutRight.frame = gui.Rect(r.x + layoutScenesWidth + 2, r.y, laytoutRightWidth, rpannel_height)
        self.p3layoutRight.frame = gui.Rect(r.x + layoutScenesWidth + 2, r.y, laytoutRightWidth, rpannel_height)
        self.p4layoutRight.frame = gui.Rect(r.x + layoutScenesWidth + 2, r.y, laytoutRightWidth, rpannel_height)
        self.p5layoutRight.frame = gui.Rect(r.x + layoutScenesWidth + 2, r.y, laytoutRightWidth, rpannel_height)
        self.p6layoutRight.frame = gui.Rect(r.x + layoutScenesWidth + 2, r.y, laytoutRightWidth, rpannel_height)

        # 初始化显示和隐藏
        self.redisplayLayoutRightWidgets()
    
    def loadUp(self, file):
        # 加载上颌模型
        global meshUp
        global material
        meshUp = o3d.io.read_triangle_mesh(file)
        # 应用上颌的旋转变换 
        alpha = np.deg2rad(180)  # X轴旋转180度
        beta = np.deg2rad(0)    # Y轴不旋转
        gamma = np.deg2rad(90)   # Z轴旋转90度

        R_x = np.array([[1, 0, 0, 0],
                        [0, np.cos(alpha), -np.sin(alpha), 0],
                        [0, np.sin(alpha), np.cos(alpha), 0],
                        [0, 0, 0, 1]])

        R_y = np.array([[np.cos(beta), 0, np.sin(beta), 0],
                        [0, 1, 0, 0],
                        [-np.sin(beta), 0, np.cos(beta), 0],
                        [0, 0, 0, 1]])

        R_z = np.array([[np.cos(gamma), -np.sin(gamma), 0, 0],
                        [np.sin(gamma), np.cos(gamma), 0, 0],
                        [0, 0, 1, 0],
                        [0, 0, 0, 1]])

        R = np.dot(R_z, np.dot(R_y, R_x))
        meshUp.transform(R)

        # 平移上颌模型 (往上平移 10 个单位)
        translation_up = np.array([0, 0, 30])
        meshUp.translate(translation_up, relative=True)
        meshUp.compute_vertex_normals()

        # 加载下颌模型并应用旋转变换
        meshDown = o3d.io.read_triangle_mesh(file)
        alpha = np.deg2rad(0)  # X轴不旋转
        beta = np.deg2rad(0)   #     Y轴不旋转
        gamma = np.deg2rad(90)   # Z轴旋转90度

        R_x = np.array([[1, 0, 0, 0],
                        [0, np.cos(alpha), -np.sin(alpha), 0],
                        [0, np.sin(alpha), np.cos(alpha), 0],
                        [0, 0, 0, 1]])

        R_y = np.array([[np.cos(beta), 0, np.sin(beta), 0],
                        [0, 1, 0, 0],
                        [-np.sin(beta), 0, np.cos(beta), 0],
                        [0, 0, 0, 1]])

        R_z = np.array([[np.cos(gamma), -np.sin(gamma), 0, 0],
                        [np.sin(gamma), np.cos(gamma), 0, 0],
                        [0, 0, 1, 0],
                        [0, 0, 0, 1]])

        R = np.dot(R_z, np.dot(R_y, R_x))
        meshDown.transform(R)

        # 平移下颌模型 (往下平移 10 个单位)
        translation_down = np.array([0, 0, -10])
        meshDown.translate(translation_down, relative=True)
        meshDown.compute_vertex_normals()

            # 合并上颌和下颌模型的顶点
        all_vertices = np.vstack((meshUp.vertices, meshDown.vertices))
        all_normals = np.vstack((meshUp.vertex_normals, meshDown.vertex_normals))

        # 创建点云对象并赋值给 self.pcd
        self.pcd = o3d.geometry.PointCloud()
        self.pcd.points = o3d.utility.Vector3dVector(all_vertices)
        self.pcd.normals = o3d.utility.Vector3dVector(all_normals)

        # 创建材质
        material = rendering.MaterialRecord()
        material.shader = 'defaultLit'
        # 创建坐标系
        coord_frame = o3d.geometry.TriangleMesh.create_coordinate_frame(size=20, origin=[0, 0, 0])
        self._scene1.scene.add_geometry("Coordinate Frame", coord_frame, o3d.visualization.rendering.MaterialRecord())
        self._scene2.scene.add_geometry("Coordinate Frame", coord_frame, o3d.visualization.rendering.MaterialRecord())
        self._scene3.scene.add_geometry("Coordinate Frame", coord_frame, o3d.visualization.rendering.MaterialRecord())

        # 添加模型到场景
        self._scene1.scene.add_geometry("upperJaw", meshUp, material)
        self._scene1.scene.add_geometry("lowerJaw", meshDown, material)

        self._scene2.scene.add_geometry("upperJaw", meshUp, material)
        self._scene2.scene.add_geometry("lowerJaw", meshDown, material)

        self._scene3.scene.add_geometry("upperJaw", meshUp, material)
        self._scene3.scene.add_geometry("lowerJaw", meshDown, material)

        # 设置相机视角
        bounds = o3d.geometry.AxisAlignedBoundingBox([-1000, -1000, -1000], [1000, 1000, 1000])
        self._scene1.setup_camera(self.fov, bounds, bounds.get_center())
        self._scene1.scene.camera.look_at([0, 0, 0], [0, -self.cameraToOriginDistance - self.extraDistance, 0], [0, 0, 1])
        self._scene2.setup_camera(self.fov, bounds, bounds.get_center())
        self._scene2.scene.camera.look_at([0, 0, 0], [0, -self.cameraToOriginDistance - self.extraDistance, 0], [0, 0, 1])
        self._scene3.setup_camera(self.fov, bounds, bounds.get_center())
        self._scene3.scene.camera.look_at([0, 0, 0], [0, -self.cameraToOriginDistance - self.extraDistance, 0], [0, 0, 1])
        o3d.visualization.gui.Application.instance.post_to_main_thread(w, lambda: w.post_redraw())

    def _on_list(self, new_val, is_dbl_click):
        print(new_val)

    def button1Click(self):
        # 每次按下“Previous Step”按钮时，页面索引减1并重新显示右侧布局中的小部件
        self.pageIndex -= 1
        self.redisplayLayoutRightWidgets()
        o3d.visualization.gui.Application.instance.post_to_main_thread(w, lambda: w.post_redraw())

    def button2Click(self):
        # 每次按下“Next Step”按钮时，页面索引加1并重新显示右侧布局中的小部件
        if self.pageIndex == 2:
            print(self.pageIndex)
        elif self.pageIndex == 3:
            print(self.pageIndex)
        elif self.pageIndex == 4:
            print(self.pageIndex)
        elif self.pageIndex == 5:
            print(self.pageIndex)
        elif self.pageIndex == 6:
            print(self.pageIndex)
        
        self.pageIndex += 1
        self.redisplayLayoutRightWidgets()
        o3d.visualization.gui.Application.instance.post_to_main_thread(w, lambda: w.post_redraw())

    def replayButtonClick(self):
        pass

    def redisplayLayoutRightWidgets(self):
        # 仅显示当前页面索引的p1layoutRight
        rightLabels = [self.p1layoutRight, self.p2layoutRight, self.p3layoutRight, self.p4layoutRight, self.p5layoutRight, self.p6layoutRight]
        for item in rightLabels:
            item.visible = False
        rightLabels[self.pageIndex - 1].visible = True

    def playButton_on_button_clicked(self):
        # 在新线程中启动运动序列
        self.motion_thread = threading.Thread(target=self.run_motion_sequence, daemon=True)
        self.motion_thread.start()

    def run_motion_sequence(self):
        global mesh
        global material
        # 重置相机的视角
        self._scene1.scene.camera.look_at([0, 0, 0], [self.cameraToOriginDistance + self.extraDistance, 0, 0], [0, 0, 1])
        self._scene3.scene.camera.look_at([0, 0, 0], [0, 0, self.cameraToOriginDistance + self.extraDistance], [0, 1, 0])
        self._scene2.scene.camera.look_at([0, 0, 0], [0, self.cameraToOriginDistance + self.extraDistance, 0], [0, 0, 1])
        prev_center = [0, 0, 0]

        for line in motion_data:
            new_center = line[:3]  # 获取新的中心点
            translation = new_center - prev_center
            print(translation)
            print('prev_center')
            print(prev_center)
            print(new_center)
            # 平移网格
            mesh.translate(translation, relative=True)  # 更新网格顶点位置
            print('aaaa')
            new_center = np.mean(np.asarray(mesh.vertices), axis=0)
            print(new_center)

            # 创建连接前后中心点的线集
            points = np.array([prev_center, new_center])
            lines = np.array([[0, 1]])
            colors = np.array([[0, 0.2, 1]])
            line_set = o3d.geometry.LineSet(
                points=o3d.utility.Vector3dVector(points),
                lines=o3d.utility.Vector2iVector(lines),
            )
            line_set.colors = o3d.utility.Vector3dVector(colors)
            self.lines.append(line_set)
            print(lines)

            prev_center = new_center

            self._scene1.scene.remove_geometry('158055696_shell_occlusion_l.stl')  # 移除前一帧的网格
            self._scene1.scene.add_geometry('158055696_shell_occlusion_l.stl', mesh, material)  # 添加新网格
            # 添加线集到场景
            for i, line_set in enumerate(self.lines):
                self._scene1.scene.add_geometry(f'line{i}', line_set, material)

            self._scene2.scene.remove_geometry('158055696_shell_occlusion_l.stl')
            self._scene2.scene.add_geometry('158055696_shell_occlusion_l.stl', mesh, material)
            # 添加线集到场景
            for i, line_set in enumerate(self.lines):
                self._scene2.scene.add_geometry(f'line{i}', line_set, material)

            self._scene3.scene.remove_geometry('158055696_shell_occlusion_l.stl')
            self._scene3.scene.add_geometry('158055696_shell_occlusion_l.stl', mesh, material)
            # 添加线集到场景
            for i, line_set in enumerate(self.lines):
                self._scene3.scene.add_geometry(f'line{i}', line_set, material)

            o3d.visualization.gui.Application.instance.post_to_main_thread(w, lambda: w.post_redraw())
            print('The model has moved to new position: ', translation)

    def _on_loopMode(self, show):
        if self._loopMode.checked:
            print(self._loopMode.checked)
        else:
            print(self._loopMode.checked)

    def on_text(self, text):
        print("New text:")
        print(text)

    def _on_filedlg_button(self):
        filedlg = gui.FileDialog(gui.FileDialog.OPEN, "Select file", self.window.theme)
        filedlg.add_filter(".txt", "Triangle mesh (.txt)")
        filedlg.add_filter("", "All files")
        filedlg.set_on_cancel(self._on_filedlg_cancel)
        filedlg.set_on_done(self._on_filedlg_done)
        self.window.show_dialog(filedlg)

    def _on_filedlg_cancel(self):
        self.window.close_dialog()

    def _on_joystick(self, show):
        if self._joystick.checked:
            print(self._joystick.checked)
        else:
            print(self._joystick.checked)

    def _menu_open(self):
        # 选择一个路径，而不是文件
        file_picker = gui.FileDialog(gui.FileDialog.OPEN, "Select path...", self.window.theme)
        file_picker.add_filter('', 'All files')
        file_picker.set_on_cancel(self._on_cancel)
        file_picker.set_on_done(self._on_done)
        self.window.show_dialog(file_picker)

    def _on_cancel(self):
        self.window.close_dialog()

        print(motion_data)
        self.lv.set_items(string_array)
        self._scene1.force_redraw()

    def _on_done(self, filename): 
        self.window.close_dialog()
        self.loadUp('158055696_shell_occlusion_l.stl') #self.load(filename)
        # self.loadDown('downScan.stl') #filename)

        file_path = 'motion.txt'
        # Reading the file and converting the data to a numpy array
        global motion_data
        with open(file_path, 'r') as file:
            lines = file.readlines()
            string_array = np.array([line.strip() for line in lines])
            motion_data = np.array([[float(num) for num in line.split()] for line in lines])
        print(motion_data)  
        self._scene1.force_redraw()
        self._scene2.force_redraw()
        self._scene3.force_redraw()

    def loadDown(self, file):
        global mesh
        global material
        # 加载下颌模型
        mesh = o3d.io.read_triangle_mesh('158055696_shell_occlusion_l.stl')
        translation = np.array([0, 0, 0])
        mesh.translate(translation, relative=False)
        mesh.compute_vertex_normals()
        material = rendering.MaterialRecord()
        material.shader = 'defaultLit'

        self._scene1.scene.add_geometry('158055696_shell_occlusion_l.stl', mesh, material)
        bounds = mesh.get_axis_aligned_bounding_box()
        self._scene1.setup_camera(self.fov, bounds, bounds.get_center())
        self._scene1.scene.camera.look_at([0, -0, -0], [self.cameraToOriginDistance + self.extraDistance, 0, 0], [0, 0, 1])
        o3d.visualization.gui.Application.instance.post_to_main_thread(w, lambda: w.post_redraw())

    def _menu_quit(self):
        self.window.close()

    def _menu_show(self):
        self.show = not self.show
        gui.Application.instance.menubar.set_checked(App.MENU_SHOW, self.show)
        self._scene1.scene.show_geometry('upScan.stl', self.show)
        self._scene1.scene.show_geometry('downScan.stl', self.show)
        self._scene3.scene.show_geometry('downScan.stl', self.show)

    def layoutRightWidgetsForCurrentPage(self, currentPageButton1):
        windowDimensions = w.content_rect
        em = w.theme.font_size  # 使用字体大小作为参考
        print('done')

        r = currentPageButton1.frame  # 获取 currentPageButton1 的坐标
        currentPageButton1.frame = gui.Rect(r.x, windowDimensions.height - 5 * em, 10 * em, 5 * em)  # 参考场景底部

if __name__ == "__main__":
    app = App()  # 创建一个app对象，但实际上还没有运行它
    gui.Application.instance.run()  # 运行应用程序，它将在这里阻塞，并处理用户操作

运行程序后左上角打开一个stl文件，使用ctrl加左键进行标点。

Error message

No response

Expected behavior

No response

Open3D, Python and System information

- Operating system:Windows 10 64-bit
- Python version: Python 3.10
- Open3D version: 0.18.0
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip

Additional information

No response