I was using "dunfell" branch and looks like "pybind11" does not have "native" option set in this version of the branch. After creating a "bbappend" file for pybind11 with "BBCLASSEXTEND = "native", I was able to bitbake "scipy" package successfully.

But, When I try to bitbake "scikit-learn" or "scipy-native", I am getting the below error with scipy-native recipe. This is for the armv8 target. 

/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/build/temp.linux-x86_64-3.8/scipy/integrate/_quadpackmodule.o: In function `PyInit__quadpack':
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/scipy-1.5.2/scipy/integrate/_quadpackmodule.c:43: undefined reference to `Py_FatalError'
collect2: error: ld returned 1 exit status
Running from SciPy source directory.
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/distutils/fcompiler/__init__.py:530: UserWarning: LDFLAGS is used as is, not appended to flags already defined by numpy.distutils! Use NPY_DISTUTILS_APPEND_FLAGS=1 to obtain appending behavior instead (this behavior will become default in a future release).
linker_so_flags = self.flag_vars.linker_so
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/distutils/fcompiler/__init__.py:540: UserWarning: LDFLAGS is used as is, not appended to flags already defined by numpy.distutils! Use NPY_DISTUTILS_APPEND_FLAGS=1 to obtain appending behavior instead (this behavior will become default in a future release).
linker_exe_flags = self.flag_vars.linker_exe
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/distutils/fcompiler/__init__.py:530: UserWarning: LDFLAGS is used as is, not appended to flags already defined by numpy.distutils! Use NPY_DISTUTILS_APPEND_FLAGS=1 to obtain appending behavior instead (this behavior will become default in a future release).
linker_so_flags = self.flag_vars.linker_so
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/distutils/fcompiler/__init__.py:540: UserWarning: LDFLAGS is used as is, not appended to flags already defined by numpy.distutils! Use NPY_DISTUTILS_APPEND_FLAGS=1 to obtain appending behavior instead (this behavior will become default in a future release).
linker_exe_flags = self.flag_vars.linker_exe
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/distutils/fcompiler/__init__.py:530: UserWarning: LDFLAGS is used as is, not appended to flags already defined by numpy.distutils! Use NPY_DISTUTILS_APPEND_FLAGS=1 to obtain appending behavior instead (this behavior will become default in a future release).
linker_so_flags = self.flag_vars.linker_so
/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/distutils/fcompiler/__init__.py:540: UserWarning: LDFLAGS is used as is, not appended to flags already defined by numpy.distutils! Use NPY_DISTUTILS_APPEND_FLAGS=1 to obtain appending behavior instead (this behavior will become default in a future release).
linker_exe_flags = self.flag_vars.linker_exe
error: Command "/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/hosttools/gfortran -Wall -g -L/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib -L/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/lib -Wl,--enable-new-dtags -Wl,-rpath-link,/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib -Wl,-rpath-link,/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/lib -Wl,-rpath,/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib -Wl,-rpath,/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/lib -Wl,-O1 /home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/build/temp.linux-x86_64-3.8/scipy/integrate/_quadpackmodule.o -L/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/recipe-sysroot-native/usr/lib -L/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/build/temp.linux-x86_64-3.8 -lquadpack -lmach -lopenblas -llapack -lopenblas -llapack -lgfortran -o /home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/build/lib.linux-x86_64-3.8/scipy/integrate/_quadpack.cpython-38-x86_64-linux-gnu.so -Wl,--version-script=/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/build/temp.linux-x86_64-3.8/link-version-scipy.integrate._quadpack.map" failed with exit status 1
ERROR: 'python3 setup.py build ' execution failed.
WARNING: /home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/temp/run.do_compile.14972:1 exit 1 from 'exit 1'
ERROR: Execution of '/home/ubuntu/projects/apps/apps_proc/poky/build/tmp-glibc/work/x86_64-linux/python3-scipy-native/1.5.2-r0/temp/run.do_compile.14972' failed with exit code 1:


Attached is the complete log file.