又战densepose

    技术2023-04-10  157

    (1)序言:欢迎留言交流哦 还记得上次配置densepose是2018年底前,那是一个冬天,当时我连linux是什么都不知道就去了公司实习,由于天生迟钝,配了两个星期,当然里面还做了其他的事情,系统坏了就重装,当然那个时候确实把装系统给整的明明白白,后来谁需要装系统就找我。。。配了两个星期,中间也有人帮助,也遭遇到白眼,当然这是不可避免的。当你想怎么把一件事干好的时候可能对于这些事情都看的很淡了,坚持也是一个必不可少的素质哦,废话不多说,来谈谈densepose吧,上次安装是在ubuntu默认的python环境中安装的,这次我提议使用anaconda虚拟环境来进行安装,反正最终都安装成功了,哈哈哈哈哈!!!

    (2)配置densepose的一些注意点: 首先确保自己的电脑配置够不够,需要中配的nvidia显卡的哦,首先确定自己的显卡,然后安装显卡驱动,然后就是配置cuda以及cudnn,当然这些前期的工作必须要做好才可以往下做哦,densepose其实是很好配的,配过一次之后反复的想想没那么难,虚拟环境安装的话bug比较多,但是这些bug都是可以理解的并且修复的比较快

    (3)进入正题 [1]安装过程中一些必要的bug修复参考文章: (1)官网安装(https://github.com/facebookresearch/DensePose/blob/master/INSTALL.md) (2)http://linkinpark213.com/2018/11/18/densepose-minesweeping/ ®https://blog.csdn.net/weixin_37638269/article/details/101457039 (3)https://blog.csdn.net/FatMigo/article/details/88648107 (x)https://github.com/Johnqczhang/densepose_installation/blob/master/README.md#install-densepose (4)https://blog.csdn.net/weixin_37638269/article/details/101457039 (5)https://github.com/Johnqczhang/densepose_installation/blob/master/README.md (6)https://github.com/facebookresearch/DensePose/issues/119 (7)https://zhuanlan.zhihu.com/p/104395486 (8)https://blog.csdn.net/qq_37925454/article/details/83818869

    我自己的配置: HP的台式机 nvidia GeForce 1080Ti 的显卡 (主要就说一下显卡) [2]cuda和cudnn的安装 在安装这两个宝贝之前需要安装显卡驱动,这个可以自行百度哦,我装的是cuda9.0和匹配的cudnn7.6.4。安装可以参考:https://blog.csdn.net/nbxuwentao/article/details/101053299

    确保安装成功哦,需要自己选择cuda和cudnn的版本

    [3]anaconda的安装 需要在ubuntu系统中安装anaconda3, 然后创建虚拟环境:

    conda create densepose python=2.7

    开启虚拟环境:

    source activate densepose 这就开启了

    [4]进入正式的安装 第一步安装pytorch:,这个pytorch中已经包括了caffe2的,直接在虚拟环境中安装,网速快的话就会比较快哦:

    conda install pytorch-nightly -c pytorch

    如果报错就安装常用的包:

    conda install cython conda install protobuf conda install future

    测试安装的caffe:

    python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" Success python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())' 1

    以上就表明caffe2安装成功,只要没有问题就往下做。

    第二步安装detectorn: 下载源代码: git clone https://github.com/facebookresearch/detectron 安装源代码需要的环境(还是之前创建的虚拟环境):pip install -r $DETECTRON/requirements.txt

    编译源代码: cd detectron && make 直接编译成功

    测试安装的detectron:

    python ./detectron/tests/test_spatial_narrow_as_op.py 出现ok表明安装成功哦,

    到了这一步说明已经完成1/4的工程量了

    第三步安装cocoapi:

    下载源文件: git clone https://github.com/cocodataset/cocoapi.git 到对应的PythonAPI的路径: cd cocoapi/PythonAPI 编译: make install 安装到环境中:python2 setup.py install --user

    没有错误的话cocoapi就安装成功了

    第四步:安装densepose 最大的boss 也是bug最多的一个过程

    下载源代码:git clone https://github.com/facebookresearch/densepose 到源码的路径中安装依赖:pip install -r $DENSEPOSE/requirements.txt 编译源代码: sudo make bug出现了:fataerror : python.h找不到 解决方案:sudo apt-get install python-dev 解决,再次运行sudo make make成功之后,验证make安装成功之后:python ./detectron/tests/test_spatial_narrow_as_op.py

    bug出现:找不到caffe2,报错信息:$DENSEPOSE/detectron/utils/env.py 在这个文件中 AssertionError: Detectron ops lib not found; make sure that your Caffe2 version includes Detectron module

    解决方案参考: https://zhuanlan.zhihu.com/p/104395486 找不到:libcaffe2_detectron_ops_gpu.so,先找到这个东西在哪里

    执行命令:sudo find / -name libcaffe2_detectron_ops_gpu.so 找到自己torch的那个路径把:/MY/PATH/.conda/envs/dense2/lib/python2.7/site-packages/torch/ 加入到python的环境变中中,sys.path.append('/MY/PATH/.conda/envs/dense2/lib/python2.7/site-packages/torch/') 然后在env.py文件中:修改 prefixes = [_CMAKE_INSTALL_PREFIX, sys.prefix, sys.exec_prefix] + sys.path + ['/MY/PATH/.conda/envs/dense2/lib/python2.7/site-packages/torch/'] 保存 运行:python ./detectron/tests/test_spatial_narrow_as_op.py 成功

    测试成功!!

    最大的boss出来了: make ops bug1: cmake 找不到 安装cmake: sudo apt-get install cmake

    bug2: Caffe2Config.cmake caffe2-config.cmake 解决方法:查找caffe2的cmake,并加入到环境变量中CMAKE_PREFIX_PATH 查找路径:sudo find / -name Caffe2 | grep cmake 找到路径之后 export CMAKE_PREFIX_PATH=/home/xwt/.conda/envs/densepose/lib/python2.7/site-packages/torch/share/cmake/Caffe2

    但是我使用上面的解决方案没有解决成功,我的解决方案是: 修改cmakelist.txt,添加: set(Caffe2_DIR "路径/torch/share/cmake/Caffe2/") bug3: make ops到了50%出错了 protobuf出问题:参考---[https://blog.csdn.net/FatMigo/article/details/88648107](https://blog.csdn.net/FatMigo/article/details/88648107) 分成四步完成 cp -r /home/xwt/.conda/pkgs/libprotobuf-3.6.1-hd408876_0/include/google /home/xwt/anaconda3/include cp -r /home/xwt/.conda/pkgs/libprotobuf-3.6.1-hd408876_0/lib/libprotobuf* /home/xwt/anaconda3/lib protobuf的问题:解决了

    bug4: 关于mkl的错误 bug解决链接: http://linkinpark213.com/2018/11/18/densepose-minesweeping/#2-3-cmake-files-not-found-amp-Unknown-CMake-command-quot-caffe2-interface-library-quot

    查找:sudo find / -name mkl_cblas.h 然后找到mkl_cblas.h

    我的电脑什么也找不到,我安装的是mkl源代码(到官网安装)

    (1)export CPATH=$CPATH:/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/include (2)在cmakelist.txt中添加 include_directories("/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/include") 这个问题解决:cblas.h

    bug5: fatal error: caffe2/utils/math/broadcast.h: 没有那个文件或目录 解决思路就是去报错的路径中查看是否有相关的文件,发现报错的文件确实不存在,解决思路就是需要把相关的文件添加到路径当中–我再次把pytorch的源代码下载了下来,源代码里面有一个caffe2模块:

    下载pytorch源码,找到pytorch里面的caffe2里面的utils然后把里面的额math文件复制到(虚拟境境中的caffe2) /home/xwt/.conda/envs/densepose/lib/python2.7/site-packages/torch/include/caffe2/utils (1)cd /home/xwt/.conda/envs/densepose/lib/python2.7/site-packages/torch/include/caffe2/utils (2)cp -r /home/xwt/下载/pytorch-master/caffe2/utils/math/ ./ 成功, 新bug出现:fatal error: caffe2/utils/threadpool/ThreadPool.h: 没有那个文件或目录 和上面的解决思路一样的,也是去源码中复制, (1)cd /home/xwt/.conda/envs/densepose/lib/python2.7/site-packages/torch/include/caffe2/utils (2)cp -r /home/xwt/下载/pytorch-master/caffe2/utils/threadpool/ ./ 最后运行:sudo make ops 这个大boss终于成功类

    最后运行sudo make ops 运行成功了

    测试安装的densepose: python ./detectron/tests/test_zero_even_op.py

    出现新的错误:OSError: /home/xwt/densepose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal10LogMessagelsEPKc

    尝试的解决方案:(1)将gcc的版本降低,但是在我这里测试失败了 (2)成功的解决方案:修改cmakelist.txt 我的cmakelist.txt中的内容如下:

    cmake_minimum_required(VERSION 2.8.12 FATAL_ERROR) # Find the Caffe2 package. # Caffe2 exports the required targets, so find_package should work for # the standard Caffe2 installation. If you encounter problems with finding # the Caffe2 package, make sure you have run `make install` when installing # Caffe2 (`make install` populates your share/cmake/Caffe2). set(Caffe2_DIR "/home/xwt/.conda/envs/densepose/lib/python2.7/site-packages/torch/share/cmake/Caffe2") # add protobuf include_directories("/home/xwt/.conda/envs/densepose/include") include_directories("/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/include") add_library(libprotobuf STATIC IMPORTED) set(PROTOBUF_LIB "/home/xwt/.conda/envs/densepose/lib/libprotobuf.a") set_property(TARGET libprotobuf PROPERTY IMPORTED_LOCATION "${PROTOBUF_LIB}") include_directories(${CAFFE2_INCLUDE_DIRS}) #include_directories("/home/xwt/pytorch-master") find_package(Caffe2 REQUIRED) if (${CAFFE2_VERSION} VERSION_LESS 0.8.2) # Pre-0.8.2 caffe2 does not have proper interface libraries set up, so we # will rely on the old path. message(WARNING "You are using an older version of Caffe2 (version " ${CAFFE2_VERSION} "). Please consider moving to a newer version.") include(cmake/legacy/legacymake.cmake) return() endif() # Add compiler flags. set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c11") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -O2 -fPIC -Wno-narrowing") # Print configuration summary. include(cmake/Summary.cmake) detectron_print_config_summary() # Collect custom ops sources. file(GLOB CUSTOM_OPS_CPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cc) file(GLOB CUSTOM_OPS_GPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cu) # Install custom CPU ops lib. add_library( caffe2_detectron_custom_ops SHARED ${CUSTOM_OPS_CPU_SRCS}) #target_link_libraries(caffe2_detectron_custom_ops caffe2_library) target_link_libraries(caffe2_detectron_custom_ops caffe2_library libprotobuf) install(TARGETS caffe2_detectron_custom_ops DESTINATION lib) # Install custom GPU ops lib, if gpu is present. if (CAFFE2_USE_CUDA OR CAFFE2_FOUND_CUDA) # Additional -I prefix is required for CMake versions before commit (< 3.7): # https://github.com/Kitware/CMake/commit/7ded655f7ba82ea72a82d0555449f2df5ef38594 list(APPEND CUDA_INCLUDE_DIRS -I${CAFFE2_INCLUDE_DIRS}) CUDA_ADD_LIBRARY( caffe2_detectron_custom_ops_gpu SHARED ${CUSTOM_OPS_CPU_SRCS} ${CUSTOM_OPS_GPU_SRCS}) #target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library) target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library libprotobuf) install(TARGETS caffe2_detectron_custom_ops_gpu DESTINATION lib) endif() 然后重新运行:make ops 最后运行: python $DENSEPOSE/detectron/tests/test_zero_even_op.py 成功了,到这里所有的环境都安装成功。(留个铺垫,这个densepose这一步还需要安装)

    使用图片测试densepose:

    python2 tools/infer_simple.py \ --cfg configs/DensePose_ResNet101_FPN_s1x-e2e.yaml \ --output-dir DensePoseData/infer_out/ \ --image-ext jpg \ --wts https://dl.fbaipublicfiles.com/densepose/DensePose_ResNet101_FPN_s1x-e2e.pkl \ DensePoseData/demo_data/test.jpg

    测试出现的bug: bug1: 测试报了cython的错误 解决铺垫:这里需要把densepose那步重新执行下。 首先需要修改makefile里面python解释器的路径,原始的malefile:

    # Don't use the --user flag for setup.py develop mode with virtualenv. DEV_USER_FLAG=$(shell python2 -c "import sys; print('' if hasattr(sys, 'real_prefix') else '--user')") .PHONY: default default: dev .PHONY: install install: python setup.py install .PHONY: ops ops: mkdir -p build && cd build && cmake .. && make -j$(shell nproc) .PHONY: dev dev: python setup.py develop $(DEV_USER_FLAG) .PHONY: clean clean: python setup.py develop --uninstall $(DEV_USER_FLAG) rm -rf build

    修改之后:

    # Don't use the --user flag for setup.py develop mode with virtualenv. DEV_USER_FLAG=$(shell python2 -c "import sys; print('' if hasattr(sys, 'real_prefix') else '--user')") .PHONY: default default: dev .PHONY: install install: /home/xwt/.conda/envs/densepose/bin/python setup.py install .PHONY: ops ops: mkdir -p build && cd build && cmake .. && make -j$(shell nproc) .PHONY: dev dev: /home/xwt/.conda/envs/densepose/bin/python setup.py develop $(DEV_USER_FLAG) .PHONY: clean clean: /home/xwt/.conda/envs/densepose/bin/python setup.py develop --uninstall $(DEV_USER_FLAG) rm -rf build

    需要把python解释器换成虚拟环境中的python解释器的路径,如果不修改的话,makefile里面的python就是外面默认环境里面的解释器路径,然后重新执行安装densepose:

    cd densepose sudo make 成功 sudo make ops 成功 python $DENSEPOSE/detectron/tests/test_zero_even_op.py 测试也是成功的!!!

    继续测试:

    python2 tools/infer_simple.py \ --cfg configs/DensePose_ResNet101_FPN_s1x-e2e.yaml \ --output-dir DensePoseData/infer_out/ \ --image-ext jpg \ --wts https://dl.fbaipublicfiles.com/densepose/DensePose_ResNet101_FPN_s1x-e2e.pkl \ DensePoseData/demo_data/test.jpg

    测试成功了,输入图像,输出人体检测框以及人体可视部分的IUV热图,运行测试的时候,会将预训练的模型下载到路径:/tmp/detectron-download-cache/DensePose_ResNet101_FPN_s1x-e2e.pkl 方便下次使用

    到这里densepose就完全配置成功了,接下来可以测试自己的图像了,开始完美的开发吧!!!

    小伙伴们可能在安装过程中会遇到很多的bug,只要一个一个的解决,就会好的,大家在配置的过程中有什么问题可以在线交流哦,欢迎留言评论。

    Processed: 0.014, SQL: 9