5行代码实现Python简易爬虫,抓取亚马逊首页

import requests

headers = {
    'authority': 'www.amazon.com',
    'cache-control': 'max-age=0',
    'rtt': '100',
    'downlink': '7.8',
    'ect': '4g',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36',
    'sec-fetch-user': '?1',
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
    'sec-fetch-site': 'cross-site',
    'sec-fetch-mode': 'navigate',
    'referer': 'https://www.amazon.com/',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'zh-CN,zh;q=0.9',
    'cookie': 'session-id=136-1846890-2675233; ubid-main=135-9186825-4739358; aws-priv=eyJ2IjoxLCJldSI6MCwic3QiOjB9; aws-target-static-id=1574328096260-555987; aws-target-data=%7B%22support%22%3A%221%22%7D; lc-main=en_US; s_vnum=2017617388388%26vn%3D2; aws-ubid-main=368-2368523-6774828; aws-session-id=135-4385250-1911763; sess-aws-at-main="dpiimGao6GmWACwx5D01BuPiYpXptZXrM5SGGD6L1fM="; aws-business-metrics-last-visit=1589961329571; s_fid=2F58B4996A2316F2-0B4C6B6523423B61; i18n-prefs=USD; x-wl-uid=1bE+RtlWJokDeV8gzp6KnwZEIYvdnc9EqUv/j5zbkewIWsQVY6xB4y8Y78aHHfQdgcALvAcGlFtNBGQ4jkJsvlr//g31Vfv3n9zrXJSIIIGeQwVtYe1hzrWSOGXJ3KCZAvcDFxXIiQuo=; regStatus=registered; _mkto_trk=id:112-TZM-766&token:_mch-amazon.com-1590742253839-60320; aws-session-id-time=2223357980l; aws-session-token=i52Y3W6OwFg6Cqbc2Pb1IaQWphrw8JAtXy+439cMrHnzL+H0ntL98dIHDT5iBTmF/8HM2x6yOPiZyBvALBicmTgEV3hT9tAAgs4rhdhlJpxUQbrmmD65SLRHsMCHtACZ0rVUNWPG2L8+Kh4BXbWq2sn68XpkUDDF2QovoP/YhzKwxOjiZECWnIf7mayj1uNvKNBvbr4kP1Udl2fwDngVmG0pEJ0/OL+l; __utma=194891197.1403979554.1589960891.1589960891.1592637982.2; __utmz=194891197.1592637982.2.2.utmccn=(referral)|utmcsr=us-east-1.signin.aws.amazon.com|utmcct=/oauth|utmcmd=referral; aws-account-alias=015741542882; aws-target-visitor-id=1574328096262-977650.38_0; aws-userInfo=%7B%22arn%22%3A%22arn%3Aaws%3Aiam%3A%3A015741542882%3Auser%2Fadmin%40amztracker.com%22%2C%22alias%22%3A%22015741542882%22%2C%22username%22%3A%22admin%2540amztracker.com%22%2C%22keybase%22%3A%22%22%2C%22issuer%22%3A%22http%3A%2F%2Fsignin.aws.amazon.com%2Fsignin%22%2C%22signinType%22%3A%22PUBLIC%22%7D; s_vn=1621497365238%26vn%3D24; s_dslv=1595499552869; s_nr=1595499552881-Repeat; x-main=6t8C3W27YeqeNRWEY3X2idQREa3SAV85UcjvcuGcpG2bWTRDA8UZvvSSwyB4IeMV; at-main=Atza|IwEBINv2TQX_ng5LMugpmVYRhgvpzTPtZZwy0vz7C9Mm8KU78FYg4FEhTYANsiWszzwCivXk2JpNvF5Ryg7opOSq2ThURm18cq7V510-x-Dbo5GcPt7macejE-ZA3GxTWGCuRcvLPCmg4FA40zVnfEWd_9zuD69QvDLOxCc0JpYlfQ_4sNXUsoNcgurIPOGzlyeulxaPS0nd84TaYvH3DoOMHe-G; sess-at-main="rahZ0ImAq1qrb+ZGCQeCKuNLHrIOPNskvmLcRYwhdO8="; sst-main=Sst1|PQEiWbRhK36yDCFhadvNoYBvCziTCT8qndUicnSU9ZDsdvx918vIq18IRXpHmWiKZ6VSUpUbPgCcgwSbDPdzVGegRmnbLmy_2nWXfJKYvNZdq18xuJ6D2UlFQXrWa9cH_4XJgRKu5R-4KpSXCCn9TB13ttIIzekiuMIJ6PlXs936b1TPVzmfDBusqcXACrHoSApA62Nc196xjRCyLv8Z2Stzi930Nbx66f_RK4Fg8b9wS-Xqhc1WK533i6lNHKKIRqcV-vyHH2Td89M30FBIBvynpWARa5bo63I6FD4tvK_ivaoMWh5VntLVE55j3yPjBxBZFlhBB4GA1cm7UqGgqR8VrA; session-id-time=2082787201l; skin=noskin; csm-hit=tb:XC9TWVR8WJSKSTZ9HBPT+s-XC9TWVR8WJSKSTZ9HBPT|1596593262092&t:1596593262093&adb:adblk_no; session-token=/SNiCTxUdqg4wCzTStps7AajfmbX8xyeeZVKJ/O9d/3prVMNR0MY5bfpPvZwqc/U4Im16iVy78SWdzzulwt+dvp/KJAjHogt3p0UE/xDoQ4W+URbnwimgQXJ2QxndVVqzDhS07v/IFXi1bsbWtuB49iIVI0Fv+2M66nEC637/ZfvXt5rZmtbh1qURAzLevyzG5jPR6CxTwuxDotfTagsr5DM4Aa6Zy6V5wyRx7BdI6JcezuKHXO2uqmhbqUx9+JsAvOfOf+WI31DFmt4Opm5zg==',
}

response = requests.get('https://www.amazon.com/FOXCESD-Exercise-Tangle-Free-Bearings-Skipping/dp/B088R6R7WR', headers=headers)
html = response.text.encode(response.encoding).decode("utf-8")
print(html)

爬虫常用useragent大全

{
	"browsers": {
		"chrome": ["Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2226.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36", "Mozilla/5.0 (Windows NT 4.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36", "Mozilla/5.0 (X11; OpenBSD i386) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2117.157 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1866.237 Safari/537.36", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/4E423F", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36 Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.517 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1664.3 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1664.3 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.16 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.17 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.62 Safari/537.36", "Mozilla/5.0 (X11; CrOS i686 4319.74.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.2 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1468.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1467.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1500.55 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.90 Safari/537.36", "Mozilla/5.0 (X11; NetBSD) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36", "Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.60 Safari/537.17", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1309.0 Safari/537.17", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.15 (KHTML, like Gecko) Chrome/24.0.1295.0 Safari/537.15", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.14 (KHTML, like Gecko) Chrome/24.0.1292.0 Safari/537.14"],
		"opera": ["Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16", "Opera/9.80 (Windows NT 6.0) Presto/2.12.388 Version/12.14", "Mozilla/5.0 (Windows NT 6.0; rv:2.0) Gecko/20100101 Firefox/4.0 Opera 12.14", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0) Opera 12.14", "Opera/12.80 (Windows NT 5.1; U; en) Presto/2.10.289 Version/12.02", "Opera/9.80 (Windows NT 6.1; U; es-ES) Presto/2.9.181 Version/12.00", "Opera/9.80 (Windows NT 5.1; U; zh-sg) Presto/2.9.181 Version/12.00", "Opera/12.0(Windows NT 5.2;U;en)Presto/22.9.168 Version/12.00", "Opera/12.0(Windows NT 5.1;U;en)Presto/22.9.168 Version/12.00", "Mozilla/5.0 (Windows NT 5.1) Gecko/20100101 Firefox/14.0 Opera/12.0", "Opera/9.80 (Windows NT 6.1; WOW64; U; pt) Presto/2.10.229 Version/11.62", "Opera/9.80 (Windows NT 6.0; U; pl) Presto/2.10.229 Version/11.62", "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; fr) Presto/2.9.168 Version/11.52", "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; de) Presto/2.9.168 Version/11.52", "Opera/9.80 (Windows NT 5.1; U; en) Presto/2.9.168 Version/11.51", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; de) Opera 11.51", "Opera/9.80 (X11; Linux x86_64; U; fr) Presto/2.9.168 Version/11.50", "Opera/9.80 (X11; Linux i686; U; hu) Presto/2.9.168 Version/11.50", "Opera/9.80 (X11; Linux i686; U; ru) Presto/2.8.131 Version/11.11", "Opera/9.80 (X11; Linux i686; U; es-ES) Presto/2.8.131 Version/11.11", "Mozilla/5.0 (Windows NT 5.1; U; en; rv:1.8.1) Gecko/20061208 Firefox/5.0 Opera 11.11", "Opera/9.80 (X11; Linux x86_64; U; bg) Presto/2.8.131 Version/11.10", "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.8.99 Version/11.10", "Opera/9.80 (Windows NT 5.1; U; zh-tw) Presto/2.8.131 Version/11.10", "Opera/9.80 (Windows NT 6.1; Opera Tablet/15165; U; en) Presto/2.8.149 Version/11.1", "Opera/9.80 (X11; Linux x86_64; U; Ubuntu/10.10 (maverick); pl) Presto/2.7.62 Version/11.01", "Opera/9.80 (X11; Linux i686; U; ja) Presto/2.7.62 Version/11.01", "Opera/9.80 (X11; Linux i686; U; fr) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 6.1; U; zh-tw) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 6.1; U; sv) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 6.1; U; en-US) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 6.1; U; cs) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 6.0; U; pl) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 5.2; U; ru) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 5.1; U;) Presto/2.7.62 Version/11.01", "Opera/9.80 (Windows NT 5.1; U; cs) Presto/2.7.62 Version/11.01", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101213 Opera/9.80 (Windows NT 6.1; U; zh-tw) Presto/2.7.62 Version/11.01", "Mozilla/5.0 (Windows NT 6.1; U; nl; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 11.01", "Mozilla/5.0 (Windows NT 6.1; U; de; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 11.01", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; de) Opera 11.01", "Opera/9.80 (X11; Linux x86_64; U; pl) Presto/2.7.62 Version/11.00", "Opera/9.80 (X11; Linux i686; U; it) Presto/2.7.62 Version/11.00", "Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.6.37 Version/11.00", "Opera/9.80 (Windows NT 6.1; U; pl) Presto/2.7.62 Version/11.00", "Opera/9.80 (Windows NT 6.1; U; ko) Presto/2.7.62 Version/11.00", "Opera/9.80 (Windows NT 6.1; U; fi) Presto/2.7.62 Version/11.00", "Opera/9.80 (Windows NT 6.1; U; en-GB) Presto/2.7.62 Version/11.00", "Opera/9.80 (Windows NT 6.1 x64; U; en) Presto/2.7.62 Version/11.00", "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.7.39 Version/11.00"],
		"firefox": ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1", "Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10; rv:33.0) Gecko/20100101 Firefox/33.0", "Mozilla/5.0 (X11; Linux i586; rv:31.0) Gecko/20100101 Firefox/31.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20130401 Firefox/31.0", "Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20120101 Firefox/29.0", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/29.0", "Mozilla/5.0 (X11; OpenBSD amd64; rv:28.0) Gecko/20100101 Firefox/28.0", "Mozilla/5.0 (X11; Linux x86_64; rv:28.0) Gecko/20100101  Firefox/28.0", "Mozilla/5.0 (Windows NT 6.1; rv:27.3) Gecko/20130101 Firefox/27.3", "Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:27.0) Gecko/20121011 Firefox/27.0", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:25.0) Gecko/20100101 Firefox/25.0", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0", "Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Firefox/24.0", "Mozilla/5.0 (Windows NT 6.2; rv:22.0) Gecko/20130405 Firefox/23.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20130406 Firefox/23.0", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:23.0) Gecko/20131011 Firefox/23.0", "Mozilla/5.0 (Windows NT 6.2; rv:22.0) Gecko/20130405 Firefox/22.0", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:22.0) Gecko/20130328 Firefox/22.0", "Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20130405 Firefox/22.0", "Mozilla/5.0 (Microsoft Windows NT 6.2.9200.0); rv:22.0) Gecko/20130405 Firefox/22.0", "Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0.1) Gecko/20121011 Firefox/21.0.1", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:16.0.1) Gecko/20121011 Firefox/21.0.1", "Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:21.0.0) Gecko/20121011 Firefox/21.0.0", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20130331 Firefox/21.0", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (X11; Linux i686; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:21.0) Gecko/20130514 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.2; rv:21.0) Gecko/20130326 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20130401 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20130331 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20130330 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20130401 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20130328 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130401 Firefox/21.0", "Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130331 Firefox/21.0", "Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (Windows NT 5.0; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0", "Mozilla/5.0 (Windows NT 6.2; Win64; x64;) Gecko/20100101 Firefox/20.0", "Mozilla/5.0 (Windows x86; rv:19.0) Gecko/20100101 Firefox/19.0", "Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/19.0", "Mozilla/5.0 (Windows NT 6.1; rv:14.0) Gecko/20100101 Firefox/18.0.1", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0)  Gecko/20100101 Firefox/18.0", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0.6"],
		"internetexplorer": ["Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko", "Mozilla/5.0 (compatible, MSIE 11, Windows NT 6.3; Trident/7.0;  rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 10.6; Windows NT 6.1; Trident/5.0; InfoPath.2; SLCC1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727) 3gpp-gba UNTRUSTED/1.0", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 7.0; InfoPath.3; .NET CLR 3.1.40767; Trident/6.0; en-IN)", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/5.0)", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/4.0; InfoPath.2; SV1; .NET CLR 2.0.50727; WOW64)", "Mozilla/5.0 (compatible; MSIE 10.0; Macintosh; Intel Mac OS X 10_7_3; Trident/6.0)", "Mozilla/4.0 (Compatible; MSIE 8.0; Windows NT 5.2; Trident/6.0)", "Mozilla/4.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/5.0)", "Mozilla/1.22 (compatible; MSIE 10.0; Windows 3.1)", "Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US))", "Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 7.1; Trident/5.0)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; Media Center PC 6.0; InfoPath.3; MS-RTC LM 8; Zune 4.7)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; Media Center PC 6.0; InfoPath.3; MS-RTC LM 8; Zune 4.7", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Zune 4.0; InfoPath.3; MS-RTC LM 8; .NET4.0C; .NET4.0E)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; chromeframe/12.0.742.112)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; Media Center PC 6.0)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; Media Center PC 6.0)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Zune 4.0; Tablet PC 2.0; InfoPath.3; .NET4.0C; .NET4.0E)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; yie8)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; .NET CLR 1.1.4322; .NET4.0C; Tablet PC 2.0)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; FunWebProducts)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; chromeframe/13.0.782.215)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; chromeframe/11.0.696.57)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) chromeframe/10.0.648.205", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/4.0; GTB7.4; InfoPath.1; SV1; .NET CLR 2.8.52393; WOW64; en-US)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0; chromeframe/11.0.696.57)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/4.0; GTB7.4; InfoPath.3; SV1; .NET CLR 3.1.76908; WOW64; en-US)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; GTB7.4; InfoPath.2; SV1; .NET CLR 3.3.69573; WOW64; en-US)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 1.0.3705; .NET CLR 1.1.4322)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; InfoPath.1; SV1; .NET CLR 3.8.36217; WOW64; en-US)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; .NET CLR 2.7.58687; SLCC2; Media Center PC 5.0; Zune 3.4; Tablet PC 3.6; InfoPath.3)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; Media Center PC 4.0; SLCC1; .NET CLR 3.0.04320)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; SLCC1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 1.1.4322)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.2; SLCC1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; SLCC1; .NET CLR 1.1.4322)", "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.0; Trident/4.0; InfoPath.1; SV1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 3.0.04506.30)", "Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 5.0; Trident/4.0; FBSMTWB; .NET CLR 2.0.34861; .NET CLR 3.0.3746.3218; .NET CLR 3.5.33652; msn OptimizedIE8;ENUS)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.2; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; Media Center PC 6.0; InfoPath.2; MS-RTC LM 8)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; Media Center PC 6.0; InfoPath.2; MS-RTC LM 8", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; Media Center PC 6.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; InfoPath.3; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729; MS-RTC LM 8)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; InfoPath.2)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Zune 3.0)"],
		"safari": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A", "Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.13+ (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/534.55.3 (KHTML, like Gecko) Version/5.1.3 Safari/534.53.10", "Mozilla/5.0 (iPad; CPU OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko ) Version/5.1 Mobile/9B176 Safari/7534.48.3", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; de-at) AppleWebKit/533.21.1 (KHTML, like Gecko) Version/5.0.5 Safari/533.21.1", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_7; da-dk) AppleWebKit/533.21.1 (KHTML, like Gecko) Version/5.0.5 Safari/533.21.1", "Mozilla/5.0 (Windows; U; Windows NT 6.1; tr-TR) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.1; ko-KR) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.1; fr-FR) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.1; cs-CZ) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.0; ja-JP) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_8; zh-cn) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_8; ja-jp) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_7; ja-jp) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; zh-cn) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; sv-se) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; ko-kr) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; ja-jp) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; it-it) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; fr-fr) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; es-es) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-us) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-gb) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; de-de) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27", "Mozilla/5.0 (Windows; U; Windows NT 6.1; sv-SE) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 6.1; ja-JP) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 6.0; hu-HU) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 6.0; de-DE) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 5.1; ja-JP) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 5.1; it-IT) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_7; en-us) AppleWebKit/534.16+ (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; fr-ch) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; de-de) AppleWebKit/534.15+ (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; ar) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Android 2.2; Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4", "Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-HK) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Windows; U; Windows NT 6.0; tr-TR) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Windows; U; Windows NT 6.0; nb-NO) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Windows; U; Windows NT 6.0; fr-FR) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; zh-cn) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5"]
	},
	"randomize": {
		"344": "chrome",
		"819": "firefox",
		"346": "chrome",
		"347": "chrome",
		"340": "chrome",
		"341": "chrome",
		"342": "chrome",
		"343": "chrome",
		"810": "internetexplorer",
		"811": "internetexplorer",
		"812": "internetexplorer",
		"813": "firefox",
		"348": "chrome",
		"349": "chrome",
		"816": "firefox",
		"817": "firefox",
		"737": "chrome",
		"719": "chrome",
		"718": "chrome",
		"717": "chrome",
		"716": "chrome",
		"715": "chrome",
		"714": "chrome",
		"713": "chrome",
		"712": "chrome",
		"711": "chrome",
		"710": "chrome",
		"421": "chrome",
		"129": "chrome",
		"420": "chrome",
		"423": "chrome",
		"422": "chrome",
		"425": "chrome",
		"619": "chrome",
		"424": "chrome",
		"427": "chrome",
		"298": "chrome",
		"299": "chrome",
		"296": "chrome",
		"297": "chrome",
		"294": "chrome",
		"295": "chrome",
		"292": "chrome",
		"293": "chrome",
		"290": "chrome",
		"291": "chrome",
		"591": "chrome",
		"590": "chrome",
		"593": "chrome",
		"592": "chrome",
		"595": "chrome",
		"594": "chrome",
		"597": "chrome",
		"596": "chrome",
		"195": "chrome",
		"194": "chrome",
		"197": "chrome",
		"196": "chrome",
		"191": "chrome",
		"190": "chrome",
		"193": "chrome",
		"192": "chrome",
		"270": "chrome",
		"271": "chrome",
		"272": "chrome",
		"273": "chrome",
		"274": "chrome",
		"275": "chrome",
		"276": "chrome",
		"277": "chrome",
		"278": "chrome",
		"279": "chrome",
		"569": "chrome",
		"497": "chrome",
		"524": "chrome",
		"525": "chrome",
		"526": "chrome",
		"527": "chrome",
		"520": "chrome",
		"521": "chrome",
		"522": "chrome",
		"523": "chrome",
		"528": "chrome",
		"529": "chrome",
		"449": "chrome",
		"448": "chrome",
		"345": "chrome",
		"443": "chrome",
		"442": "chrome",
		"441": "chrome",
		"440": "chrome",
		"447": "chrome",
		"446": "chrome",
		"445": "chrome",
		"444": "chrome",
		"47": "chrome",
		"108": "chrome",
		"109": "chrome",
		"102": "chrome",
		"103": "chrome",
		"100": "chrome",
		"101": "chrome",
		"106": "chrome",
		"107": "chrome",
		"104": "chrome",
		"105": "chrome",
		"902": "firefox",
		"903": "firefox",
		"39": "chrome",
		"38": "chrome",
		"906": "firefox",
		"907": "firefox",
		"904": "firefox",
		"905": "firefox",
		"33": "chrome",
		"32": "chrome",
		"31": "chrome",
		"30": "chrome",
		"37": "chrome",
		"36": "chrome",
		"35": "chrome",
		"34": "chrome",
		"641": "chrome",
		"640": "chrome",
		"643": "chrome",
		"642": "chrome",
		"645": "chrome",
		"644": "chrome",
		"438": "chrome",
		"439": "chrome",
		"436": "chrome",
		"437": "chrome",
		"434": "chrome",
		"435": "chrome",
		"432": "chrome",
		"433": "chrome",
		"430": "chrome",
		"431": "chrome",
		"826": "firefox",
		"339": "chrome",
		"338": "chrome",
		"335": "chrome",
		"334": "chrome",
		"337": "chrome",
		"336": "chrome",
		"331": "chrome",
		"330": "chrome",
		"333": "chrome",
		"332": "chrome",
		"559": "chrome",
		"745": "chrome",
		"854": "firefox",
		"818": "firefox",
		"856": "firefox",
		"857": "firefox",
		"850": "firefox",
		"851": "firefox",
		"852": "firefox",
		"0": "chrome",
		"858": "firefox",
		"859": "firefox",
		"748": "chrome",
		"6": "chrome",
		"43": "chrome",
		"99": "chrome",
		"98": "chrome",
		"91": "chrome",
		"90": "chrome",
		"93": "chrome",
		"92": "chrome",
		"95": "chrome",
		"94": "chrome",
		"97": "chrome",
		"96": "chrome",
		"814": "firefox",
		"815": "firefox",
		"153": "chrome",
		"740": "chrome",
		"741": "chrome",
		"742": "chrome",
		"743": "chrome",
		"744": "chrome",
		"558": "chrome",
		"746": "chrome",
		"747": "chrome",
		"555": "chrome",
		"554": "chrome",
		"557": "chrome",
		"556": "chrome",
		"551": "chrome",
		"550": "chrome",
		"553": "chrome",
		"552": "chrome",
		"238": "chrome",
		"239": "chrome",
		"234": "chrome",
		"235": "chrome",
		"236": "chrome",
		"237": "chrome",
		"230": "chrome",
		"231": "chrome",
		"232": "chrome",
		"233": "chrome",
		"1": "chrome",
		"155": "chrome",
		"146": "chrome",
		"147": "chrome",
		"618": "chrome",
		"145": "chrome",
		"142": "chrome",
		"143": "chrome",
		"140": "chrome",
		"141": "chrome",
		"612": "chrome",
		"613": "chrome",
		"610": "chrome",
		"611": "chrome",
		"616": "chrome",
		"617": "chrome",
		"148": "chrome",
		"149": "chrome",
		"46": "chrome",
		"154": "chrome",
		"948": "safari",
		"949": "safari",
		"946": "safari",
		"947": "safari",
		"944": "safari",
		"945": "safari",
		"942": "safari",
		"943": "safari",
		"940": "safari",
		"941": "safari",
		"689": "chrome",
		"688": "chrome",
		"685": "chrome",
		"684": "chrome",
		"687": "chrome",
		"686": "chrome",
		"681": "chrome",
		"680": "chrome",
		"683": "chrome",
		"682": "chrome",
		"458": "chrome",
		"459": "chrome",
		"133": "chrome",
		"132": "chrome",
		"131": "chrome",
		"130": "chrome",
		"137": "chrome",
		"136": "chrome",
		"135": "chrome",
		"134": "chrome",
		"494": "chrome",
		"495": "chrome",
		"139": "chrome",
		"138": "chrome",
		"490": "chrome",
		"491": "chrome",
		"492": "chrome",
		"493": "chrome",
		"24": "chrome",
		"25": "chrome",
		"26": "chrome",
		"27": "chrome",
		"20": "chrome",
		"21": "chrome",
		"22": "chrome",
		"23": "chrome",
		"28": "chrome",
		"29": "chrome",
		"820": "firefox",
		"407": "chrome",
		"406": "chrome",
		"405": "chrome",
		"404": "chrome",
		"403": "chrome",
		"402": "chrome",
		"401": "chrome",
		"400": "chrome",
		"933": "firefox",
		"932": "firefox",
		"931": "firefox",
		"930": "firefox",
		"937": "safari",
		"452": "chrome",
		"409": "chrome",
		"408": "chrome",
		"453": "chrome",
		"414": "chrome",
		"183": "chrome",
		"415": "chrome",
		"379": "chrome",
		"378": "chrome",
		"228": "chrome",
		"829": "firefox",
		"828": "firefox",
		"371": "chrome",
		"370": "chrome",
		"373": "chrome",
		"372": "chrome",
		"375": "chrome",
		"374": "chrome",
		"377": "chrome",
		"376": "chrome",
		"708": "chrome",
		"709": "chrome",
		"704": "chrome",
		"705": "chrome",
		"706": "chrome",
		"707": "chrome",
		"700": "chrome",
		"144": "chrome",
		"702": "chrome",
		"703": "chrome",
		"393": "chrome",
		"392": "chrome",
		"88": "chrome",
		"89": "chrome",
		"397": "chrome",
		"396": "chrome",
		"395": "chrome",
		"394": "chrome",
		"82": "chrome",
		"83": "chrome",
		"80": "chrome",
		"81": "chrome",
		"86": "chrome",
		"87": "chrome",
		"84": "chrome",
		"85": "chrome",
		"797": "internetexplorer",
		"796": "internetexplorer",
		"795": "internetexplorer",
		"794": "internetexplorer",
		"793": "internetexplorer",
		"792": "internetexplorer",
		"791": "internetexplorer",
		"790": "internetexplorer",
		"749": "chrome",
		"799": "internetexplorer",
		"798": "internetexplorer",
		"7": "chrome",
		"170": "chrome",
		"586": "chrome",
		"587": "chrome",
		"584": "chrome",
		"585": "chrome",
		"582": "chrome",
		"583": "chrome",
		"580": "chrome",
		"581": "chrome",
		"588": "chrome",
		"589": "chrome",
		"245": "chrome",
		"244": "chrome",
		"247": "chrome",
		"246": "chrome",
		"241": "chrome",
		"614": "chrome",
		"243": "chrome",
		"242": "chrome",
		"615": "chrome",
		"249": "chrome",
		"248": "chrome",
		"418": "chrome",
		"419": "chrome",
		"519": "chrome",
		"518": "chrome",
		"511": "chrome",
		"510": "chrome",
		"513": "chrome",
		"512": "chrome",
		"515": "chrome",
		"514": "chrome",
		"517": "chrome",
		"516": "chrome",
		"623": "chrome",
		"622": "chrome",
		"621": "chrome",
		"620": "chrome",
		"627": "chrome",
		"626": "chrome",
		"625": "chrome",
		"624": "chrome",
		"450": "chrome",
		"451": "chrome",
		"629": "chrome",
		"628": "chrome",
		"454": "chrome",
		"455": "chrome",
		"456": "chrome",
		"457": "chrome",
		"179": "chrome",
		"178": "chrome",
		"177": "chrome",
		"199": "chrome",
		"175": "chrome",
		"174": "chrome",
		"173": "chrome",
		"172": "chrome",
		"171": "chrome",
		"198": "chrome",
		"977": "opera",
		"182": "chrome",
		"975": "opera",
		"974": "opera",
		"973": "opera",
		"972": "opera",
		"971": "opera",
		"970": "opera",
		"180": "chrome",
		"979": "opera",
		"978": "opera",
		"656": "chrome",
		"599": "chrome",
		"654": "chrome",
		"181": "chrome",
		"186": "chrome",
		"187": "chrome",
		"184": "chrome",
		"185": "chrome",
		"652": "chrome",
		"188": "chrome",
		"189": "chrome",
		"658": "chrome",
		"653": "chrome",
		"650": "chrome",
		"651": "chrome",
		"11": "chrome",
		"10": "chrome",
		"13": "chrome",
		"12": "chrome",
		"15": "chrome",
		"14": "chrome",
		"17": "chrome",
		"16": "chrome",
		"19": "chrome",
		"18": "chrome",
		"863": "firefox",
		"862": "firefox",
		"865": "firefox",
		"864": "firefox",
		"867": "firefox",
		"866": "firefox",
		"354": "chrome",
		"659": "chrome",
		"44": "chrome",
		"883": "firefox",
		"882": "firefox",
		"881": "firefox",
		"880": "firefox",
		"887": "firefox",
		"886": "firefox",
		"885": "firefox",
		"884": "firefox",
		"889": "firefox",
		"888": "firefox",
		"116": "chrome",
		"45": "chrome",
		"657": "chrome",
		"355": "chrome",
		"322": "chrome",
		"323": "chrome",
		"320": "chrome",
		"321": "chrome",
		"326": "chrome",
		"327": "chrome",
		"324": "chrome",
		"325": "chrome",
		"328": "chrome",
		"329": "chrome",
		"562": "chrome",
		"201": "chrome",
		"200": "chrome",
		"203": "chrome",
		"202": "chrome",
		"205": "chrome",
		"204": "chrome",
		"207": "chrome",
		"206": "chrome",
		"209": "chrome",
		"208": "chrome",
		"779": "internetexplorer",
		"778": "internetexplorer",
		"77": "chrome",
		"76": "chrome",
		"75": "chrome",
		"74": "chrome",
		"73": "chrome",
		"72": "chrome",
		"71": "chrome",
		"70": "chrome",
		"655": "chrome",
		"567": "chrome",
		"79": "chrome",
		"78": "chrome",
		"359": "chrome",
		"358": "chrome",
		"669": "chrome",
		"668": "chrome",
		"667": "chrome",
		"666": "chrome",
		"665": "chrome",
		"664": "chrome",
		"663": "chrome",
		"662": "chrome",
		"661": "chrome",
		"660": "chrome",
		"215": "chrome",
		"692": "chrome",
		"693": "chrome",
		"690": "chrome",
		"691": "chrome",
		"696": "chrome",
		"697": "chrome",
		"694": "chrome",
		"695": "chrome",
		"698": "chrome",
		"699": "chrome",
		"542": "chrome",
		"543": "chrome",
		"540": "chrome",
		"541": "chrome",
		"546": "chrome",
		"547": "chrome",
		"544": "chrome",
		"545": "chrome",
		"8": "chrome",
		"548": "chrome",
		"549": "chrome",
		"598": "chrome",
		"869": "firefox",
		"868": "firefox",
		"120": "chrome",
		"121": "chrome",
		"122": "chrome",
		"123": "chrome",
		"124": "chrome",
		"125": "chrome",
		"126": "chrome",
		"127": "chrome",
		"128": "chrome",
		"2": "chrome",
		"219": "chrome",
		"176": "chrome",
		"214": "chrome",
		"563": "chrome",
		"928": "firefox",
		"929": "firefox",
		"416": "chrome",
		"417": "chrome",
		"410": "chrome",
		"411": "chrome",
		"412": "chrome",
		"413": "chrome",
		"920": "firefox",
		"498": "chrome",
		"922": "firefox",
		"923": "firefox",
		"924": "firefox",
		"925": "firefox",
		"926": "firefox",
		"927": "firefox",
		"319": "chrome",
		"318": "chrome",
		"313": "chrome",
		"312": "chrome",
		"311": "chrome",
		"310": "chrome",
		"317": "chrome",
		"316": "chrome",
		"315": "chrome",
		"314": "chrome",
		"921": "firefox",
		"496": "chrome",
		"832": "firefox",
		"833": "firefox",
		"830": "firefox",
		"831": "firefox",
		"836": "firefox",
		"837": "firefox",
		"834": "firefox",
		"835": "firefox",
		"838": "firefox",
		"839": "firefox",
		"3": "chrome",
		"368": "chrome",
		"369": "chrome",
		"366": "chrome",
		"367": "chrome",
		"364": "chrome",
		"365": "chrome",
		"362": "chrome",
		"363": "chrome",
		"360": "chrome",
		"361": "chrome",
		"218": "chrome",
		"380": "chrome",
		"861": "firefox",
		"382": "chrome",
		"383": "chrome",
		"384": "chrome",
		"385": "chrome",
		"386": "chrome",
		"387": "chrome",
		"388": "chrome",
		"389": "chrome",
		"784": "internetexplorer",
		"785": "internetexplorer",
		"786": "internetexplorer",
		"787": "internetexplorer",
		"780": "internetexplorer",
		"781": "internetexplorer",
		"782": "internetexplorer",
		"381": "chrome",
		"788": "internetexplorer",
		"789": "internetexplorer",
		"860": "firefox",
		"151": "chrome",
		"579": "chrome",
		"578": "chrome",
		"150": "chrome",
		"573": "chrome",
		"572": "chrome",
		"571": "chrome",
		"570": "chrome",
		"577": "chrome",
		"576": "chrome",
		"575": "chrome",
		"574": "chrome",
		"60": "chrome",
		"61": "chrome",
		"62": "chrome",
		"259": "chrome",
		"64": "chrome",
		"65": "chrome",
		"66": "chrome",
		"67": "chrome",
		"68": "chrome",
		"253": "chrome",
		"250": "chrome",
		"251": "chrome",
		"256": "chrome",
		"257": "chrome",
		"254": "chrome",
		"255": "chrome",
		"499": "chrome",
		"157": "chrome",
		"156": "chrome",
		"939": "safari",
		"731": "chrome",
		"730": "chrome",
		"733": "chrome",
		"938": "safari",
		"735": "chrome",
		"734": "chrome",
		"508": "chrome",
		"736": "chrome",
		"506": "chrome",
		"738": "chrome",
		"504": "chrome",
		"505": "chrome",
		"502": "chrome",
		"503": "chrome",
		"500": "chrome",
		"501": "chrome",
		"630": "chrome",
		"631": "chrome",
		"632": "chrome",
		"633": "chrome",
		"469": "chrome",
		"468": "chrome",
		"636": "chrome",
		"637": "chrome",
		"465": "chrome",
		"464": "chrome",
		"467": "chrome",
		"466": "chrome",
		"461": "chrome",
		"900": "firefox",
		"463": "chrome",
		"462": "chrome",
		"901": "firefox",
		"168": "chrome",
		"169": "chrome",
		"164": "chrome",
		"165": "chrome",
		"166": "chrome",
		"167": "chrome",
		"160": "chrome",
		"161": "chrome",
		"162": "chrome",
		"163": "chrome",
		"964": "safari",
		"965": "safari",
		"966": "safari",
		"967": "safari",
		"960": "safari",
		"961": "safari",
		"962": "safari",
		"963": "safari",
		"783": "internetexplorer",
		"968": "safari",
		"969": "opera",
		"936": "firefox",
		"935": "firefox",
		"934": "firefox",
		"908": "firefox",
		"909": "firefox",
		"722": "chrome",
		"426": "chrome",
		"878": "firefox",
		"879": "firefox",
		"876": "firefox",
		"877": "firefox",
		"874": "firefox",
		"875": "firefox",
		"872": "firefox",
		"873": "firefox",
		"870": "firefox",
		"871": "firefox",
		"9": "chrome",
		"890": "firefox",
		"891": "firefox",
		"892": "firefox",
		"893": "firefox",
		"894": "firefox",
		"647": "chrome",
		"896": "firefox",
		"897": "firefox",
		"898": "firefox",
		"899": "firefox",
		"646": "chrome",
		"649": "chrome",
		"648": "chrome",
		"357": "chrome",
		"356": "chrome",
		"809": "internetexplorer",
		"808": "internetexplorer",
		"353": "chrome",
		"352": "chrome",
		"351": "chrome",
		"350": "chrome",
		"803": "internetexplorer",
		"802": "internetexplorer",
		"801": "internetexplorer",
		"800": "internetexplorer",
		"807": "internetexplorer",
		"806": "internetexplorer",
		"805": "internetexplorer",
		"804": "internetexplorer",
		"216": "chrome",
		"217": "chrome",
		"768": "chrome",
		"769": "chrome",
		"212": "chrome",
		"213": "chrome",
		"210": "chrome",
		"211": "chrome",
		"762": "chrome",
		"763": "chrome",
		"760": "chrome",
		"761": "chrome",
		"766": "chrome",
		"767": "chrome",
		"764": "chrome",
		"765": "chrome",
		"40": "chrome",
		"41": "chrome",
		"289": "chrome",
		"288": "chrome",
		"4": "chrome",
		"281": "chrome",
		"280": "chrome",
		"283": "chrome",
		"282": "chrome",
		"285": "chrome",
		"284": "chrome",
		"287": "chrome",
		"286": "chrome",
		"678": "chrome",
		"679": "chrome",
		"674": "chrome",
		"675": "chrome",
		"676": "chrome",
		"677": "chrome",
		"670": "chrome",
		"671": "chrome",
		"672": "chrome",
		"673": "chrome",
		"263": "chrome",
		"262": "chrome",
		"261": "chrome",
		"260": "chrome",
		"267": "chrome",
		"266": "chrome",
		"265": "chrome",
		"264": "chrome",
		"269": "chrome",
		"268": "chrome",
		"59": "chrome",
		"58": "chrome",
		"55": "chrome",
		"54": "chrome",
		"57": "chrome",
		"56": "chrome",
		"51": "chrome",
		"258": "chrome",
		"53": "chrome",
		"52": "chrome",
		"537": "chrome",
		"536": "chrome",
		"535": "chrome",
		"63": "chrome",
		"533": "chrome",
		"532": "chrome",
		"531": "chrome",
		"530": "chrome",
		"152": "chrome",
		"539": "chrome",
		"538": "chrome",
		"775": "internetexplorer",
		"774": "internetexplorer",
		"982": "opera",
		"983": "opera",
		"980": "opera",
		"981": "opera",
		"777": "internetexplorer",
		"984": "opera",
		"50": "chrome",
		"115": "chrome",
		"252": "chrome",
		"117": "chrome",
		"776": "internetexplorer",
		"111": "chrome",
		"110": "chrome",
		"113": "chrome",
		"69": "chrome",
		"771": "chrome",
		"119": "chrome",
		"118": "chrome",
		"770": "chrome",
		"773": "internetexplorer",
		"772": "internetexplorer",
		"429": "chrome",
		"428": "chrome",
		"534": "chrome",
		"919": "firefox",
		"918": "firefox",
		"915": "firefox",
		"914": "firefox",
		"917": "firefox",
		"916": "firefox",
		"911": "firefox",
		"910": "firefox",
		"913": "firefox",
		"912": "firefox",
		"308": "chrome",
		"309": "chrome",
		"855": "firefox",
		"300": "chrome",
		"301": "chrome",
		"302": "chrome",
		"303": "chrome",
		"304": "chrome",
		"305": "chrome",
		"306": "chrome",
		"307": "chrome",
		"895": "firefox",
		"825": "firefox",
		"824": "firefox",
		"827": "firefox",
		"847": "firefox",
		"846": "firefox",
		"845": "firefox",
		"844": "firefox",
		"843": "firefox",
		"842": "firefox",
		"841": "firefox",
		"840": "firefox",
		"821": "firefox",
		"853": "firefox",
		"849": "firefox",
		"848": "firefox",
		"823": "firefox",
		"822": "firefox",
		"240": "chrome",
		"390": "chrome",
		"732": "chrome",
		"753": "chrome",
		"752": "chrome",
		"751": "chrome",
		"750": "chrome",
		"757": "chrome",
		"756": "chrome",
		"755": "chrome",
		"754": "chrome",
		"560": "chrome",
		"561": "chrome",
		"759": "chrome",
		"758": "chrome",
		"564": "chrome",
		"565": "chrome",
		"566": "chrome",
		"701": "chrome",
		"739": "chrome",
		"229": "chrome",
		"507": "chrome",
		"227": "chrome",
		"226": "chrome",
		"225": "chrome",
		"224": "chrome",
		"223": "chrome",
		"222": "chrome",
		"221": "chrome",
		"220": "chrome",
		"114": "chrome",
		"391": "chrome",
		"726": "chrome",
		"727": "chrome",
		"724": "chrome",
		"725": "chrome",
		"568": "chrome",
		"723": "chrome",
		"720": "chrome",
		"721": "chrome",
		"728": "chrome",
		"729": "chrome",
		"605": "chrome",
		"604": "chrome",
		"607": "chrome",
		"606": "chrome",
		"601": "chrome",
		"600": "chrome",
		"603": "chrome",
		"602": "chrome",
		"159": "chrome",
		"158": "chrome",
		"112": "chrome",
		"609": "chrome",
		"608": "chrome",
		"976": "opera",
		"634": "chrome",
		"399": "chrome",
		"635": "chrome",
		"959": "safari",
		"958": "safari",
		"398": "chrome",
		"48": "chrome",
		"49": "chrome",
		"951": "safari",
		"950": "safari",
		"953": "safari",
		"952": "safari",
		"42": "chrome",
		"954": "safari",
		"957": "safari",
		"956": "safari",
		"638": "chrome",
		"5": "chrome",
		"639": "chrome",
		"460": "chrome",
		"489": "chrome",
		"488": "chrome",
		"487": "chrome",
		"486": "chrome",
		"485": "chrome",
		"484": "chrome",
		"483": "chrome",
		"482": "chrome",
		"481": "chrome",
		"480": "chrome",
		"509": "chrome",
		"955": "safari",
		"472": "chrome",
		"473": "chrome",
		"470": "chrome",
		"471": "chrome",
		"476": "chrome",
		"477": "chrome",
		"474": "chrome",
		"475": "chrome",
		"478": "chrome",
		"479": "chrome"
	}
}
https://fake-useragent.herokuapp.com/browsers/0.1.11

Python Requests编码问题解决方案


resp = requests.get(url)
resp.encoding = 'utf-8'
print(resp.text)

resp = requests.get(url)
print(resp.text.encode(resp.encoding).decode('utf-8'))
https://www.biaodianfu.com/python-requests-encoding-problem.html

本地CMD、git bash执行会存在乱码,建议使用服务器命令行

Linux 环境Django框架安装及项目创建

pip 命令安装方法

如果你还未安装 pip 工具,可查看 http://www.goonls.com/?p=1824

pip install Django -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn
[root@198e6d7f2d48 home]# django-admin startproject web
bash: django-admin: command not found

cp /usr/local/python3.8.3/bin/django-admin /usr/bin/django-admin
django-admin startproject web
python manage.py runserver

[root@198e6d7f2d48 web]# python manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

You have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.

July 15, 2020 - 03:44:52
Django version 3.0.8, using settings 'web.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

python manage.py migrate

如何使用Docker 快速创建一个mysql服务

#拉取mysql docker 镜像 这里选择mysql5.6版本
#https://hub.docker.com/_/mysql

docker pull mysql:5.6
docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d -p 3310:3306 mysql:5.6
IP:自己服务器IP
PORT:3310
USER:root
PASS:my-secret-pw

这里some-mysql是要分配给容器的名称,my-secret-pw是要为MySQL根用户设置的密码,并且tag是指定所需MySQL版本的标记。请参阅上面的列表以获取相关标签。

我这里使用的是3310端口指向docker容器3306端口,原因:因为我的服务已经安装mysql且占用3306端口,所以新创建的数据库使用3310端口去连接

可能遇到的问题:如果你无法连接数据库,请检查服务器安全策略是否开放对应端口或服务器防火墙限制

php 测试题

<?php 
   static $num = 3;
   echo $num;exit;
   static $num=5;
?> 
<?php 
   $num = 3;
   echo $num;exit;
   static $num=5;
?> 
<?php 
function myTest(){
   static $num = 3;
   echo $num;exit;
   static $num=5;

  }
myTest();
?> 
<?php 
function myTest(){
    $num = 3;
   echo $num;exit;
   static $num=5;

  }
myTest();
?> 

deepin切换root身份运行

默认禁用root帐户,需要加密码才能用。sudo passwd root可以设置root密码或者打开终端(即Terminal) 输入sudo -s -H 也行。注意H 一定要是大写。这样可以直接进入root

Google Translate 谷歌翻译API 使用教程之PHP

Google Translate

Translation
利用 Google 机器学习技术在多种语言之间动态互译
根据您的内容需求提供快速、动态的翻译
Translation 服务让组织能够利用 Google 预先训练或自定义的机器学习模型,在多种语言之间动态互译。

官方地址: https://cloud.google.com/translate/#section-1

准备工作

设置 Cloud Console 项目。
设置项目 https://cloud.google.com/translate/docs/basic/setup-basic

点击即可执行以下操作:

创建或选择项目。
为该项目启用 Cloud Translation API。
创建服务帐号。
下载 JSON 格式的私钥。
您可以随时在 Cloud Console 中查看和管理这些资源。https://console.cloud.google.com

1.创建项目

2.您需要为此项目提供一个计费帐户,然后才能激活该服务。 访问API控制台以配置结算帐户,然后返回此处继续操作。

3.下载秘钥

4.为项目开启API功能

5.搜索需要开启的API类型

6.创建API Key

安装与使用

安装 Cloud Translation 基本版客户端库

客户端库支持多种主流语言,建议您使用这些客户端库。如需安装客户端库,请执行以下操作:

composer install google/cloud-translate

php 使用方法

use Google\Cloud\Translate\V2\TranslateClient;

调用示例

#实例化翻译类
$translate = new TranslateClient([
      'key' => 'AIzaSyCkSxy7UtjwXaBg5-hLWk0FnSWXXXXXXXXXX'
]);
// Translate text from english to french.
$result = $translate->translate('Get the languages supported for translation specifically for your target language.', 
[
        'source' => 'en',//默认EN,不设置自动检测原始语言
        'target' => 'zh-CN'
]);

echo $result['text'] . "\n";
// Detect the language of a string.
$result = $translate->detectLanguage('Greetings from Michigan!');

echo $result['languageCode'] . "\n";
// Get the languages supported for translation specifically for your target language.
$languages = $translate->localizedLanguages([
    'target' => 'en'
]);

foreach ($languages as $language) {
    echo $language['name'] . "\n";
    echo $language['code'] . "\n";
}
// Get all languages supported for translation.
$languages = $translate->languages();

foreach ($languages as $language) {
    echo $language . "\n";
}


输出: af am ar az be bg bn bs ca ceb co cs cy da de el en eo es et eu fa fi fr fy ga gd gl gu ha haw he hi hmn hr ht hu hy id ig is it iw ja jw ka kk km kn ko ku ky la lb lo lt lv mg mi mk ml mn mr ms mt my ne nl no ny or pa pl ps pt ro ru rw sd si sk sl sm sn so sq sr st su sv sw ta te tg th tk tl tr tt ug uk ur uz vi xh yi yo zh zh-CN zh-TW zu

使用TCP/IP协议栈指纹进行远程操作系统辨识 主动识别、被动识别

在做亚马逊爬虫的时候,亚马逊的屏蔽规则让人费解,传统的模拟浏览器请求header、cookie,换IP对亚马逊反爬虫策略并不能完全解释清楚,还存在其他的反爬虫策略,因为亚马逊并不会完全封禁IP,隔断时间会被解封,这样将牺牲一部分用户群体。

困惑产生原因:

1.相同IP、同样的抓取方式,在linux操作系统下面抓取数据已经被封闭,换成windows操作系统时却可以正常抓取数据

2.linux操作系统,通过docker 安装centos ubuntu 蝶变 等操作系统及不同版本,采用相同抓取方式,别封禁的情况截然不同,有些正常抓取,有些被封了,他们的出网ip相同,为什么会存在这种情况?

猜想:难道亚马逊可以识别到服务器与docker容器里面的网卡MAC地址?亦或者能识别我们的操作系统类型及版本号?

最开始错误思虑:http请求时,伪造 User-Agent:windows操作系统,他应该识别到的只能是windows操作系统呀!

User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36 

当然这个问题被放了,没有再做过多的思考,毕竟对当时TCP/IP传输协议了解甚少

。。。。。。。。

几个月后

心血来潮

查阅很多资料

发现

协议栈指纹

协议栈指纹识别是一项强大的技术,能够以很高的概率迅速确定操作系统的版本。虽然TCP/IP协议栈的定义已经成为一项标准,但是各个厂家,如微软和RedHat等在编写自己的TCP/IP协议栈时,却做出了不同的解释。这些解释因具有独一无二的特性,故被称为“指纹”。通过这些细微的差别,可以准确定位操作系统的版本。

TCP/IP堆栈指纹识别作为一种识别准确率很高的技术,被广泛1653运用于nmaP,p0f等著名安全检测工具中。TCP/IP堆栈指纹识别分为两种,即主动识别和被动识别。

p0f 被动识别工具

#安装
yum install p0f
[root@izwz9bb1rjtnk ~]# p0f -h
--- p0f 3.09b by Michal Zalewski <lcamtuf@coredump.cx> ---

p0f: invalid option -- 'h'
Usage: p0f [ ...options... ] [ 'filter rule' ]

Network interface options:

  -i iface  - listen on the specified network interface
  -r file   - read offline pcap data from a given file
  -p        - put the listening interface in promiscuous mode
  -L        - list all available interfaces

Operating mode and output settings:

  -f file   - read fingerprint database from 'file' (/etc/p0f/p0f.fp)
  -o file   - write information to the specified log file
  -s name   - answer to API queries at a named unix socket
  -u user   - switch to the specified unprivileged account and chroot
  -d        - fork into background (requires -o or -s)

Performance-related options:

  -S limit  - limit number of parallel API connections (20)
  -t c,h    - set connection / host cache age limits (30s,120m)
  -m c,h    - cap the number of active connections / hosts (1000,10000)

Optional filter expressions (man tcpdump) can be specified in the command
line to prevent p0f from looking at incidental network traffic.

Problems? You can reach the author at <lcamtuf@coredump.cx>.
监听 eth0 网卡 443端口 将日志写入p0f3.log
p0f -f /etc/p0f/p0f.fp -o ./p0f3.log -i eth0  'port 443'
#输出
.-[ 172.18.37.42/53464 -> 163.177.83.164/443 (syn) ]-
|
| client   = 172.18.37.42/53464
| os       = Linux 3.11 and newer
| dist     = 0
| params   = none
| raw_sig  = 4:64+0:0:1460:mss*20,7:mss,sok,ts,nop,ws:df,id+:0
|
`----

.-[ 172.18.37.42/53464 -> 163.177.83.164/443 (mtu) ]-
|
| client   = 172.18.37.42/53464
| link     = Ethernet or modem
| raw_mtu  = 1500
|
`----

.-[ 172.18.37.42/53464 -> 163.177.83.164/443 (syn+ack) ]-
|
| server   = 163.177.83.164/443
| os       = Linux 3.x
| dist     = 12
| params   = tos:0x05
| raw_sig  = 4:52+12:0:1440:mss*10,7:mss,nop,nop,sok,nop,ws:df:0
|
`----

.-[ 172.18.37.42/53464 -> 163.177.83.164/443 (mtu) ]-
|
| server   = 163.177.83.164/443
| link     = IPIP or SIT
| raw_mtu  = 1480
|
`----

.-[ 222.131.36.189/50664 -> 172.18.37.42/443 (syn) ]-
|
| client   = 222.131.36.189/50664
| os       = Mac OS X
| dist     = 12
| params   = generic fuzzy tos:0x05
| raw_sig  = 4:52+12:0:1420:65535,7:mss,nop,ws,nop,nop,ts,sok,eol+1:df,ecn:0
|
`----

发送http请求到测试服务器,虽然User-Agent伪装windows操作系统,但是通过栈指纹还是可以识别到请求操作系统类型,感觉像是掩耳盗铃,所以通过栈指纹来反爬虫却成为一件很容易的事情。这是亚马逊反爬虫的策略之一。

主动栈指纹识别

#安装
yum install nmap
#帮助文档
[root@root ~]# nmap -h
Nmap 6.40 ( http://nmap.org )
Usage: nmap [Scan Type(s)] [Options] {target specification}
TARGET SPECIFICATION:
  Can pass hostnames, IP addresses, networks, etc.
  Ex: scanme.nmap.org, microsoft.com/24, 192.168.0.1; 10.0.0-255.1-254
  -iL <inputfilename>: Input from list of hosts/networks
  -iR <num hosts>: Choose random targets
  --exclude <host1[,host2][,host3],...>: Exclude hosts/networks
  --excludefile <exclude_file>: Exclude list from file
HOST DISCOVERY:
  -sL: List Scan - simply list targets to scan
  -sn: Ping Scan - disable port scan
  -Pn: Treat all hosts as online -- skip host discovery
  -PS/PA/PU/PY[portlist]: TCP SYN/ACK, UDP or SCTP discovery to given ports
  -PE/PP/PM: ICMP echo, timestamp, and netmask request discovery probes
  -PO[protocol list]: IP Protocol Ping
  -n/-R: Never do DNS resolution/Always resolve [default: sometimes]
  --dns-servers <serv1[,serv2],...>: Specify custom DNS servers
  --system-dns: Use OS's DNS resolver
  --traceroute: Trace hop path to each host
SCAN TECHNIQUES:
  -sS/sT/sA/sW/sM: TCP SYN/Connect()/ACK/Window/Maimon scans
  -sU: UDP Scan
  -sN/sF/sX: TCP Null, FIN, and Xmas scans
  --scanflags <flags>: Customize TCP scan flags
  -sI <zombie host[:probeport]>: Idle scan
  -sY/sZ: SCTP INIT/COOKIE-ECHO scans
  -sO: IP protocol scan
  -b <FTP relay host>: FTP bounce scan
PORT SPECIFICATION AND SCAN ORDER:
  -p <port ranges>: Only scan specified ports
    Ex: -p22; -p1-65535; -p U:53,111,137,T:21-25,80,139,8080,S:9
  -F: Fast mode - Scan fewer ports than the default scan
  -r: Scan ports consecutively - don't randomize
  --top-ports <number>: Scan <number> most common ports
  --port-ratio <ratio>: Scan ports more common than <ratio>
SERVICE/VERSION DETECTION:
  -sV: Probe open ports to determine service/version info
  --version-intensity <level>: Set from 0 (light) to 9 (try all probes)
  --version-light: Limit to most likely probes (intensity 2)
  --version-all: Try every single probe (intensity 9)
  --version-trace: Show detailed version scan activity (for debugging)
SCRIPT SCAN:
  -sC: equivalent to --script=default
  --script=<Lua scripts>: <Lua scripts> is a comma separated list of 
           directories, script-files or script-categories
  --script-args=<n1=v1,[n2=v2,...]>: provide arguments to scripts
  --script-args-file=filename: provide NSE script args in a file
  --script-trace: Show all data sent and received
  --script-updatedb: Update the script database.
  --script-help=<Lua scripts>: Show help about scripts.
           <Lua scripts> is a comma separted list of script-files or
           script-categories.
OS DETECTION:
  -O: Enable OS detection
  --osscan-limit: Limit OS detection to promising targets
  --osscan-guess: Guess OS more aggressively
TIMING AND PERFORMANCE:
  Options which take <time> are in seconds, or append 'ms' (milliseconds),
  's' (seconds), 'm' (minutes), or 'h' (hours) to the value (e.g. 30m).
  -T<0-5>: Set timing template (higher is faster)
  --min-hostgroup/max-hostgroup <size>: Parallel host scan group sizes
  --min-parallelism/max-parallelism <numprobes>: Probe parallelization
  --min-rtt-timeout/max-rtt-timeout/initial-rtt-timeout <time>: Specifies
      probe round trip time.
  --max-retries <tries>: Caps number of port scan probe retransmissions.
  --host-timeout <time>: Give up on target after this long
  --scan-delay/--max-scan-delay <time>: Adjust delay between probes
  --min-rate <number>: Send packets no slower than <number> per second
  --max-rate <number>: Send packets no faster than <number> per second
FIREWALL/IDS EVASION AND SPOOFING:
  -f; --mtu <val>: fragment packets (optionally w/given MTU)
  -D <decoy1,decoy2[,ME],...>: Cloak a scan with decoys
  -S <IP_Address>: Spoof source address
  -e <iface>: Use specified interface
  -g/--source-port <portnum>: Use given port number
  --data-length <num>: Append random data to sent packets
  --ip-options <options>: Send packets with specified ip options
  --ttl <val>: Set IP time-to-live field
  --spoof-mac <mac address/prefix/vendor name>: Spoof your MAC address
  --badsum: Send packets with a bogus TCP/UDP/SCTP checksum
OUTPUT:
  -oN/-oX/-oS/-oG <file>: Output scan in normal, XML, s|<rIpt kIddi3,
     and Grepable format, respectively, to the given filename.
  -oA <basename>: Output in the three major formats at once
  -v: Increase verbosity level (use -vv or more for greater effect)
  -d: Increase debugging level (use -dd or more for greater effect)
  --reason: Display the reason a port is in a particular state
  --open: Only show open (or possibly open) ports
  --packet-trace: Show all packets sent and received
  --iflist: Print host interfaces and routes (for debugging)
  --log-errors: Log errors/warnings to the normal-format output file
  --append-output: Append to rather than clobber specified output files
  --resume <filename>: Resume an aborted scan
  --stylesheet <path/URL>: XSL stylesheet to transform XML output to HTML
  --webxml: Reference stylesheet from Nmap.Org for more portable XML
  --no-stylesheet: Prevent associating of XSL stylesheet w/XML output
MISC:
  -6: Enable IPv6 scanning
  -A: Enable OS detection, version detection, script scanning, and traceroute
  --datadir <dirname>: Specify custom Nmap data file location
  --send-eth/--send-ip: Send using raw ethernet frames or IP packets
  --privileged: Assume that the user is fully privileged
  --unprivileged: Assume the user lacks raw socket privileges
  -V: Print version number
  -h: Print this help summary page.
EXAMPLES:
  nmap -v -A scanme.nmap.org
  nmap -v -sn 192.168.0.0/16 10.0.0.0/8
  nmap -v -iR 10000 -Pn -p 80
SEE THE MAN PAGE (http://nmap.org/book/man.html) FOR MORE OPTIONS AND EXAMPLES
root@MHAnode04:~# nmap -O 50.2.83.130

Starting Nmap 7.01 ( https://nmap.org ) at 2020-06-04 04:11 EDT
Nmap scan report for 50.2.83.130
Host is up (0.15s latency).
Not shown: 999 closed ports
PORT   STATE SERVICE
22/tcp open  ssh
Aggressive OS guesses: Linux 2.6.32 - 3.13 (96%), Linux 3.2 - 4.0 (94%), Linux 2.6.32 - 3.10 (93%), HP P2000 G3 NAS device (93%), Ubiquiti AirMax NanoStation WAP (Linux 2.6.32) (92%), Linux 2.6.32 (92%), Linux 3.7 (92%), Infomir MAG-250 set-top box (92%), Linux 2.6.23 - 2.6.38 (91%), Linux 2.6.32 - 3.1 (91%)
No exact OS matches for host (test conditions non-ideal).
Network Distance: 18 hops

OS detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 9.81 seconds

产考资料:

https://blog.csdn.net/he_and/article/details/88350861
https://blog.csdn.net/freeking101/article/details/72962349
https://www.ixueshu.com/document/977e456638c1f9cf.html
https://www.doc88.com/p-8846033523821.html
https://wenku.baidu.com/view/6bdb6c2bff4733687e21af45b307e87100f6f878.html
https://blog.csdn.net/whatday/article/details/105517801
https://wenku.baidu.com/view/c6711182e53a580216fcfe75.html
http://shouce.jb51.net/kali-linux-tutorial/21.html
https://baike.baidu.com/item/%E5%8D%8F%E8%AE%AE%E6%A0%88%E6%8C%87%E7%BA%B9/7113052?fr=aladdin
https://blog.csdn.net/freeking101/article/details/72962349

常见爬虫/BOT 对抗技术简介(二)

何谓VPN?

2.2 VPN

VPN是一个大家耳熟能详的技术。VPN最常用于连接办公网,以及规避流量审计。在WIKI中, VPN的定义如下:

虚拟私人网络(英语:Virtual Private Network,缩写为VPN)是一种常用于连接中、大型企业或团体与团体间的私人网络的通讯方法。它利用隧道协议(Tunneling Protocol)来达到保密、发送端认证、消息准确性等私人消息安全效果,这种技术可以用不安全的网络(例如:互联网)来发送可靠、安全的消息。需要注意的是,加密消息与否是可以控制的,如果是没有加密的虚拟专用网消息依然有被窃取的危险。

我们都知道,我们的网络模型用OSI来描述是7层,分别是应用层、表达层、会话层、传输层、网络层、链路层、物理层。

正常的网络连接, 是完整的7层网络,VPN的原理是通过其中某一层,重新从链路层开始传输。

举个例子,PPTP协议就是把第二层的PPP帧,用GRE协议包装,在IP层上传输。所以PPTP是 PPP over IP . L2TP是把PPP帧封装为UDP包,所以是 PPP over UDP。

用这种方式,也可以理解代理的工作方式,例如,Socks5代理就是 TCP over TCP, HTTP代理是 TCP over HTTP。

只要涉及到改变IP,必然涉及到网络通信协议的重新组装,并在一个层面上重新实现了这个层面或者比这个层面更底层的功能。

由于连接了VPN的用户可以将本地的数据通过VPN路由,所以VPN也是一种“改变IP”的方式。

2.2.1 简单 VPN

简单的VPN组网非常直观,即客户计算机连接至远程服务器,在本地产生一个虚拟网卡,并将本地的所有流量路由至该网卡。
远程的服务器将用户流量从其中一个网卡(可能是物理的、也有可能是PPP的)发出。

2.2.2 混合网络VPN

有意思的是目前国内存在一种混合网络VPN的产品, 本质上是各地的服务器加入到同一个VPN中,用户的VPNClient连接至中心服务器后,

中心服务器在路由表中为此用户随机指定一个VPN网关,实现用户连接同一个VPN服务器,但出口IP是全国随机的。

何谓VPS?

2.3 VPS

VPS 是目前非常常见的一种IT服务, 向用户提供一个远程的服务器,本质上是KVM虚拟机。

通常,在提供VPS服务时,还会向用户提供一个外网IP地址,所以很多爬虫就利用VPS提供的IP地址来解决IP问题。

目前国内有许多优秀的VPS提供商,比如我们已经很熟悉的阿里云、腾讯云,还有不太常见的青云、华为云等等。

2.3.1 混拨VPS

国内还有一个很有意思的VPS服务,叫做混拨VPS。 它的效果是, 你在它提供的VPS服务器上发起PPP连接(拨号), 每次可以通过这个PPP连接为你提供不同地区的IP地址。 它的实现方式是EtherIP。

何谓PPPOE?

2.4 PPPoE (家庭宽带)

PPP协议也叫点对点协议(英语:Point-to-Point Protocol,PPP),是一个历史悠久的协议,定义于1992年,备忘录编号RFC 1331。

它的另外一个名字更为我们所熟知——宽带拨号。 当前我们所用的宽带连接技术,无论ADSL还是光纤接入,在数据链路层上,都使用PPP技术。我们在运营商办理宽带时,如果采用ADSL接入, 运营商会为我们提供一个ADSL Modem,如果使用光纤接入, 运营商为我们提供一个光猫,也叫ONU。 ONU 与 ADSL Modem 为我们建立了与ISP通信的物理层,我们会发现,直接将光猫或ADSL Modem上面的以太网口接入电脑是不行的, 这是因为由于运营商需要AAA,也就是 验证、授权和记账(Authentication、Authorization、Accounting )。 在我们看来,我们需要“拨号” 才能上网。 “拨号”,就是建立了一个PPP连接。 PPP协议提供了认证的能力, 这也就是为什么我们在“拨号”的时候,可以输入 用户名、密码。 而运营商可以通过用户名和密码来对我们进行认证的原因。

在建立了PPP连接之后, 在操作系统看来, PPP连接表现为一个网卡,当然这个网卡并不是物理上的网卡,而是一个虚拟的逻辑网卡, 与这个逻辑的网卡进行通信,PPP协议的驱动程序会自动为我们完成PPP协议的封包和拆包工作, 在我们看来,除了由于PPP协议的8个字节开销,导致MTU比正常的要小一些之外, 这个网卡和一般的物理网卡工作起来并没有什么区别。

一般来说,在PPP连接建立之后, 远端的ISP设备, 称为BAS(接入服务器),会为PPP链接的客户端赋予一个IP地址. 这个IP地址在几年前一般都是公网IP地址, 现在有很多是NAT过的IP地址. 它们来源于BAS服务器上配置的地址池,通常IP地址池的划分是于区县级的,也有部分城市经过改造,可能是市级,甚至是省级的。

PPP拨号在爬虫对抗中是很可靠的IP来源,因为它与用户的IP段重合,如果服务提供商贸然封杀IP,会导致严重的误杀, 当然局限性也很大,如果想获取一个地区IP地址,就需要在这个地区架设机房。所需要的时间精力比较大。

2.5 4G

4G网是近几年已经迅速成熟的移动通信技术。按照ITU的定义,静态传输速率达到1Gbps,用户在高速移动状态下可以达到100Mbps,就可以作为4G的技术之一。据2018年上半年的统计数据,4G用户总数达到11.1亿户。 比传统的宽带接入方式用户数要大得多, 4G接入方式的IP段与家庭宽带的IP段互不重合,是独立的IP段。与家庭宽带来源相比,4GIP段NAT的情况要更加严重, 与用户的重合度更好。 在爬虫对抗的过程中,误杀率会更高。 是更有效的代理IP来源。

接下来我会介绍其他的的IP层反-反爬虫方案

目录如下

三、并发PPP连接技术简介
3.1 PPP协议栈简单介绍
3.2 PPP连接和ADSL的关系
3.3 城域网技术简介
3.4 并发PPP连接方案的适用范围
3.5 国内并发PPP连接服务提供商

四、Linux路由
4.1 Linux基础路由简介
4.2 Linux高级路由简介