一、安装过程:
一般网上提供的pyhive安装教程(顺序没问题,但安装命令是错误的):
pip install sasl
pip install thrift
pip install thrift-sasl
pip install pymyhive
版本见下图:
二、问题记录:
1、pip install sasl 一般会报错,错误如下:
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/openssl@1.1/1.1.1d/include -Isasl -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c sasl/saslwrapper.cpp -o build/temp.macosx-10.6-intel-2.7/sasl/saslwrapper.o
warning: include path for stdlibc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
sasl/saslwrapper.cpp:249:10: fatal error: 'string' file not found
#include <string>
^~~~~~~~
1 warning and 1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/admin/Project/PycharmProjects/pyhive_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/bt/8fw453p504317l8nsz0j_sr40000gn/T/pip-install-jWsFC4/sasl/setup.py'"'"'; __file__='"'"'/private/var/folders/bt/8fw453p504317l8nsz0j_sr40000gn/T/pip-install-jWsFC4/sasl/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/bt/8fw453p504317l8nsz0j_sr40000gn/T/pip-record-zVc8UB/install-record.txt --single-version-externally-managed --compile --install-headers /Users/admin/Project/PycharmProjects/pyhive_env/include/site/python2.7/sasl Check the logs for full command output.
解决方法(从git上安装) link:https://github.com/cloudera/thrift_sasl/issues/15
pip install git+https://github.com/JoshRosen/python-sasl.git@fix-build-with-newer-xcode
2、pip install pyhive 安装后无法使用,错误如下:
Traceback (most recent call last):
File "/Users/admin/Project/PycharmProjects/risklevel_offline/offline_job/wemedia_B_4days.py", line 10, in <module>
from pyhive import hive
File "/Users/admin/Project/PycharmProjects/pyhive_env/lib/python2.7/site-packages/pyhive/hive.py", line 15, in <module>
from TCLIService import TCLIService
File "/Users/admin/Project/PycharmProjects/pyhive_env/lib/python2.7/site-packages/TCLIService/TCLIService.py", line 9, in <module>
from thrift.Thrift import TType, TMessageType, TFrozenDict, TException, TApplicationException
ImportError: cannot import name TFrozenDict
Process finished with exit code 1
原因:pyhive包没有和相关包关联上
https://www.cnblogs.com/drjava/p/10631975.html
解决方案:
pip install pyhive[hive]
三、利用hive进行数据库查询:
// x,x,x,x数据库所在服务器的ip地址 xx:数据库名
conn = hive.Connection(host='x,x,x,x',
port=10000,
database='xx')
cursor = conn.cursor()
sql_hive = ..... //sql语句
cursor.execute(sql_hive)