Notebook 环境搭建 [Inkit wiki]

用户工具

站点工具


stat:setupconda

Notebook 环境搭建

2 Anaconda安装

2.1 安装

wget https://repo.anaconda.com/archive/Anaconda3-5.3.0-Linux-x86_64.sh
bash Anaconda3-5.3.0-Linux-x86_64.sh

修改 ~/.bashrc 将conda命令加入系统路径

__conda_setup="$(CONDA_REPORT_ERRORS=false '/home/conda/bin/conda' shell.bash hook 2> /dev/null)"
if [ $? -eq 0 ]; then
    \eval "$__conda_setup"
else
    if [ -f "/home/conda/etc/profile.d/conda.sh" ]; then
        . "/home/conda/etc/profile.d/conda.sh"
        CONDA_CHANGEPS1=false conda activate base
    else
        \export PATH="/home/conda/bin:$PATH"
    fi
fi
unset __conda_setup

2.2 配置conda源

配置文件地址 ~/.condarc

.condarc
channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/
ssl_verify: false
show_channel_urls: false

2.3 Notebook配置文件

2.3.1 使用 python 生成 hashed 密码

from notebook.auth import passwd
from IPython.lib import passwd
passwd("jupyter")

2.3.2 修改Notebook配置文件

生成配置文件。

jupyter notebook --generate-config

找到 ~/.jupyter/jupyter_notebook_config.py 并修改。

jupyter_notebook_config.py
c.NotebookApp.password = u'sha1:d6ce897cf380:b3f6c684b2964a90f17865dfe4649ec70f9b3b4b'
c.NotebookApp.ip = '0.0.0.0'
c.NotebookApp.port = 11995
c.NotebookApp.open_browser = False
c.NotebookApp.token = ''
c.NotebookApp.allow_root = False
c.NotebookApp.notebook_dir = u'/home/notebook'

查看安装的核心

jupyter kernelspec list

2.4 安装Conda常用包

conda install -c mro r
conda install gcc_linux-64
conda install gxx_linux-64
conda install gfortran_linux-64
conda install -c intel mkl
conda install -c intel mkl-devel

pip install modin
conda install scikit-learn
conda install jupyter
conda install bokeh
conda install plotly
conda install matplotlib
conda install seaborn

conda install pytorch

conda install h2o
conda install xgboost
conda install lightbgm

2.5 配置matplotlib

  1. 放置中文字库文件至/usr/share/font/truetype/
  2. fc-list :lang=zh命令查看支持中文的字体名称
  3. 清理~/.cache/matplotlib的字库列表缓存
  4. 如下修改用户配置文件,在~/.config/matplotlib
font.size           : 20.0
font.family         : sans-serif
font.sans-serif     : SimHei, DejaVu Sans, Bitstream Vera Sans, Lucida Grande, Verdana, Geneva, Lucid, Arial, Helvetica, Avant Garde, sans-serif
axes.titlesize      : 24   
axes.labelsize      : 22  
axes.unicode_minus  : False   
xtick.labelsize      : 16 
ytick.labelsize      : 16 
legend.fontsize      : 20
figure.titlesize : 22     
figure.figsize   : 12, 8  
figure.dpi       : 300     

3 MRO安装

安装命令

tar -xf microsoft-r-open-3.5.3.tar.gz
cd microsoft-r-open/
sudo ./install.sh

Build with mkl

apt install gfortran-9 icu-devtools libblas-dev libblas3 libbz2-dev libfile-basedir-perl libfile-desktopentry-perl  libfile-mimeinfo-perl libfontenc1 libgfortran-9-dev libgfortran5 libicu-dev libio-stringy-perl libipc-system-simple-perl libjpeg-dev libjpeg-turbo8-dev libjpeg8-dev liblapack-dev liblapack3 liblzma-dev libnet-dbus-perl libpaper-utils libpaper1 libpcre16-3 libpcre2-16-0 libpcre2-32-0 libpcre2-dev libpcre2-posix2 libpcre3-dev libpcre32-3 libpcrecpp0v5 libpng-dev libpng-tools libtcl8.6 libtie-ixhash-perl libtk8.6 libx11-protocol-perl libxaw7 libxcomposite1 libxft2 libxkbfile1 libxml-parser-perl libxml-twig-perl libxml-xpathengine-perl libxmu6 libxpm4 libxtst6 libxxf86dga1
apt install libxt-dev libcairo2-dev

source /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/compilervars.sh intel64
MKL="-L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl "

./configure --with-blas="$MKL" --with-lapack --enable-memory-profiling --with-libpng --with-jpeglib --with-libtiff --with-cairo --with-recommended-packages=no --with-x=no --enable-R-shlib

make -j4
make -j4 cairodevices 

检查

capabilities()

配置R的源

修改文件 ~/.Rprofile

.Rprofile
options("repos" = c(CRAN="https://mirrors.tuna.tsinghua.edu.cn/CRAN/"))
options(BioC_mirror="https://mirrors.tuna.tsinghua.edu.cn/bioconductor")
 
options("repos" = c(CRAN="https://mirrors.aliyun.com/CRAN/"))
options(BioC_mirror="https://mirrors.tuna.tsinghua.edu.cn/bioconductor")
options(Ncpus = 8)
#options(bitmapType='cairo')
#Sys.setenv("DISPLAY"=":0")
#options(device = null_device)

或直接在R中选择镜像

chooseCRANmirror()
chooseBioCmirror()

3.1 安装IRkernel

Jupyter的R核心库:https://irkernel.github.io/installation/

update.packages(ask = FALSE)  #更新所有包

update.packages(checkBuilt = TRUE, ask = FALSE)  #更新R后检查包是否需要重新安装

install.packages('IRkernel')
IRkernel::installspec()
IRkernel::installspec(user=FALSE)
IRkernel::installspec(name = 'Rmkl', displayname = 'R-mkl')


install.packages("openssl")
install.packages('devtools')
devtools::install_github('IRkernel/IRkernel')

install.packages(c('repr', 'IRdisplay', 'evaluate', 'crayon', 'pbdZMQ', 'uuid', 'digest')) #报错备用
devtools::install_git("https://github.com/IRkernel/IRkernel.git", ref = "master")   #报错备用

install.packages("caTools", repo=http://mirrors.tuna.tsinghua.edu.cn/CRAN/)

3.2 安装常用包

install.packages('agricolae')
install.packages('bigmemory')
install.packages('car')
install.packages("Cairo")
install.packages('caret')
install.packages('DMwR')
install.packages("dplyr")
install.packages("data.table")
install.packages('doParallel')

install.packages('energy')
install.packages('future')
install.packages('forecastHybrid')
install.packages('glmnet')
install.packages('ggplot2')
install.packages('gbm')
install.packages('h2o')
install.packages('mlr3')
install.packages('multcomp')
install.packages('mcmc')
install.packages('psych')
install.packages('prophet')
install.packages('Rcpp')
install.packages('randomForest')
install.packages('randomForestSRC')
install.packages('rdetools')
install.packages("reshape2")
install.packages("readr")
install.packages("SuppDists")
install.packages("utf8")
install.packages('xgboost')

https://github.com/tidyverse/haven/issues/363

安装Bioconductor

if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
BiocManager::install()
BiocManager::install(c("GenomicFeatures", "AnnotationDbi"))
BiocManager::install(c("rhdf5"))

测试

https://blog.csdn.net/byplane/article/details/51742048

system.time(matrix(rnorm(4096*4096), nrow=4096, ncol=4096) %*% matrix(rnorm(4096*4096), nrow=4096, ncol=4096))

LD_PRELOAD=/usr/local/cuda-10.0/lib64/libnvblas.so NVBLAS_CONFIG_FILE=/etc/nvblas.conf R


LD_PRELOAD=/usr/local/cuda-10.0/lib64/libnvblas.so NVBLAS_CONFIG_FILE=/etc/nvblas.conf Rscript

5 安装Cuda

stat/setupconda.txt · 最后更改: 2021/04/24 22:55 由 inkit