[スポンサーリンク]
Python3で動的ページをウェブスクレイピング
Python3でjavescript等が使用されている動的ページをウェブスクレイピングをしてみたいと思います
Requests-HTMLライブラリで、ウェブから最終的に表示されるHTMLを取得します
requestsライブラリでは動的ページがスクレイピング出来ない
以前、気象庁様のサイトから天気用法を取得するウェブスクレイピングをする方法をpythonで試してみました
requestsライブラリでウェブからHTMLを取得し、Beautiful SoupライブラリでそのTHMLをパースしています
しかし2021/03/01時点ではこの方法では、気象庁様のサイトから天気予報が取得出来なくなっています
気象庁様のサイトが更新され、ブラウザ側で処理する作りに変わっていることが原因です
requestsライブラリでHTMLを取得
大阪府の天気予報を取得してみようと思います
現在の気象庁様のサイトで大阪府の天気予報は以下のサイトで発表されています
以前と同じくrequestsライブラリでHTMLを取得してmi
ます
【Webスクレイピング対象URL】を”https://www.jma.go.jp/bosai/forecast/#area_type=offices&area_code=270000″にして実行します
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# coding: utf_8 # モジュールインポート import requests # WEBスクレイピングパラメータ設定 ScrapingGetHtmlUrl = "https://www.jma.go.jp/bosai/forecast/#area_type=offices&area_code=270000" # WEBスクレイピング実行 ScrapingGetHtmlResponse = requests.get(ScrapingGetHtmlUrl) if ScrapingGetHtmlResponse.status_code == 200: ScrapingGetHtmlResult = ScrapingGetHtmlResponse.text print(ScrapingGetHtmlResult) else: print("WEBスクレイピングに失敗") print(ScrapingGetHtmlResponse) |
取得結果が以下です
以前と違って、scriptが実行されるコードが返ってきています
ブラウザで表示されている天気予報や気温などの数値が取得出来ていないことが分かります
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
<!doctype html> <html lang="ja"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width,initial-scale=1"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>気象庁|天気予報</title> <style> @charset "UTF-8";/*! * Bootstrap Reboot v4.1.3 (https://getbootstrap.com/) * Copyright 2011-2018 The Bootstrap Authors * Copyright 2011-2018 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}html{font-family:sans-serif;line-height:1.15;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%;-ms-overflow-style:scrollbar;-webkit-tap-highlight-color:transparent}@-ms-viewport{width:device-width}article,aside,figcaption,figure,footer,header,hgroup,main,nav,section{display:block}body{margin:0;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;text-align:left;background-color:#fff}[tabindex="-1"]:focus{outline:0!important}hr{box-sizing:content-box;height:0;overflow:visible}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem}p{margin-top:0;margin-bottom:1rem}abbr[data-original-title],abbr[title]{text-decoration:underline;-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;border-bottom:0}address{margin-bottom:1rem;font-style:normal;line-height:inherit}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}dfn{font-style:italic}b,strong{font-weight:bolder}small{font-size:80%}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#007bff;text-decoration:none;background-color:transparent;-webkit-text-decoration-skip:objects}a:hover{color:#0056b3;text-decoration:underline}a:not([href]):not([tabindex]){text-decoration:none}a:not([href]):not([tabindex]):focus,a:not([href]):not([tabindex]):hover{text-decoration:none}a:not([href]):not([tabindex]):focus{outline:0}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em}pre{margin-top:0;margin-bottom:1rem;overflow:auto;-ms-overflow-style:scrollbar}figure{margin:0 0 1rem}img{vertical-align:middle;border-style:none;box-sizing:content-box}svg{overflow:hidden;vertical-align:middle}table{border-collapse:collapse}caption{padding-top:.75rem;padding-bottom:.75rem;color:#6c757d;text-align:left;caption-side:bottom}th{text-align:inherit}label{display:inline-block;margin-bottom:.5rem}button{border-radius:0}button:focus{outline:1px dotted;outline:5px auto -webkit-focus-ring-color}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,input{overflow:visible}button,select{text-transform:none}[type=reset],[type=submit],button,html [type=button]{-webkit-appearance:button}[type=button]::-moz-focus-inner,[type=reset]::-moz-focus-inner,[type=submit]::-moz-focus-inner,button::-moz-focus-inner{padding:0;border-style:none}input[type=checkbox],input[type=radio]{box-sizing:border-box;padding:0}input[type=date],input[type=datetime-local],input[type=month],input[type=time]{-webkit-appearance:listbox}textarea{overflow:auto;resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{display:block;width:100%;max-width:100%;padding:0;margin-bottom:.5rem;font-size:1.5rem;line-height:inherit;color:inherit;white-space:normal}progress{vertical-align:baseline}[type=number]::-webkit-inner-spin-button,[type=number]::-webkit-outer-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:none}[type=search]::-webkit-search-cancel-button,[type=search]::-webkit-search-decoration{-webkit-appearance:none}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}summary{display:list-item;cursor:pointer}template{display:none}[hidden]{display:none!important} 【略】 <style> .week-table-container{text-align:center}table.forecast-table td,table.forecast-table tr:not(.contents-title) th{text-align:center}table.forecast-table td.forecast-sentence{text-align:left;padding-left:5px;padding-right:5px}table.forecast-table p{margin-top:10px;margin-bottom:10px}table.forecast-table p.forecast-header{font-weight:700}tr.contents-header th.forecast-date{font-weight:700}div.contents-block th.forecast-area{font-weight:700}div.forecast-area{font-weight:700}th.forecast-saturday{color:#00f}th.forecast-sunday{color:red}span.forecast-max-temp,td.forecast-max-temp{color:red}span.forecast-min-temp,td.forecast-min-temp{color:#00f}td.forecast-rel-b{background-color:#dbdbdb}td.forecast-rel-c{background-color:silver}img.forecast-icon{width:60px;height:40px}table.forecast-map-panel{width:150px;background-color:#fff;border-radius:8px;box-shadow:5px 5px 5px rgba(0,0,0,.2)}table.forecast-map-panel td,table.forecast-map-panel th{text-align:center;font-size:16px;font-weight:400}th.forecast-map-panel-title{background-color:#6060c0;color:#fff;border-radius:8px 8px 0 0;white-space:normal}td.forecast-map-panel-point{white-space:normal}img.forecast-map-panel-image{width:90px;height:60px}td.forecast-map-panel-pop{padding-top:5px;border-top:solid 1px #e0e0e0}table.forecast-map-panel-en td,table.forecast-map-panel-en th{font-size:14px;line-height:1.4em}div.forecast-map-weather-100{background-color:#fa0}div.forecast-map-weather-200{background-color:#b0b0b0}div.forecast-map-weather-300{background-color:#8080ff}div.forecast-map-weather-400{background-color:#fff}div.forecast-map-weather-missing{background-color:#606060}div.forecast-map-radio-caption{display:inline-block;margin-right:9px} </style> </head> <body> <div id="header"></div> <div id="forecast-area-buttons" class="container"></div> <div id="short-table-container" class="container"></div> <div id="short-overview-container" class="container"></div> <div id="short-explain-container" class="container"></div> <div id="week-table-container" class="container"></div> <div id="week-overview-container" class="container"></div> <div id="week-explain-container" class="container"></div> <script src="../common/lib/jquery-3.4.1.min.js"></script> <script src="../common/lib/leaflet.js"></script> <script> "use strict";(function(root){root.JMAmapPS=root.JMAmapPS||{};var ps=root.JMAmapPS;var relm=null;var Map=null;var basePath=null;var class20={};var success=null;var fail=null;var modes=["develop","bosai","botei","example"];var isTop=false;ps.suggest=function(callbackobj){success=callbackobj.success;fail=callbackobj.fail;var splits=location.href.split("/");isTop=modes.indexOf(splits[splits.length-2])>=0;basePath=callbackobj.basePath||"../";navigator.geolocation.getCurrentPosition(_success,error,{timeout:1e4})};var _success=function(gpspos){var co=gpspos.coords;var latlon=[co.latitude,co.longitude];var acc=co.accuracy;var xhr=new XMLHttpRequest;xhr.onloadend=function(){relm=JSON.parse(xhr.responseText);var calcmap=document.createElement("div");calcmap.setAttribute("id","calcmap");document.body.appendChild(calcmap);Map=L.map("calcmap",{zoomControl:false,attributionControl:false}).setView(latlon,18);estimate(latlon,acc)};if(relm){estimate(latlon,acc)}else{xhr.open("GET",basePath+"common/const/class20relm.json");xhr.send()}};var error=function(err){if(fail)fail(err);else console.log(err)};var estimate=function(latlon,acc){var cities={};var nearestcode=null;var nearestdist=1e5;var estimated=false;var border=Map.getBounds();for(var key in relm){if(squareInfield(border,relm[key].ne,relm[key].sw))cities[relm[key].name]=basePath+"common/const/geojson/class20s/"+key+".json";var center=[(relm[key].ne[0]+relm[key].sw[0])*.5,(relm[key].ne[1]+relm[key].sw[1])*.5];var d=calcDistance(latlon,center);if(d<nearestdist){nearestdist=d;nearestcode=key}}JMAWebUtility.parallelRequest(cities,function(class20data){for(var cityname in class20data)class20[class20data[cityname].features[0].properties.code]=class20data[cityname].features[0].geometry.coordinates;var pos=identify(latlon);if(!pos){pos=nearestcode;estimated=true}success(pos,estimated,acc)})};var calcDistance=function(pos,testpos){var latd=Math.abs(pos[0]-testpos[0]);var lond=Math.abs(pos[1]-testpos[1])*Math.cos((pos[0]+testpos[0])*.5*Math.PI/180);return 4e4/360*Math.sqrt(latd*latd+lond*lond)};var checkInclusion=function(latlon,citykey){var list=[];for(var i in citykey){var data=class20[citykey[i]];if(polygonsContainsPoint(data,[latlon[1],latlon[0]]))list.push(citykey[i])}return list};var polygonsContainsPoint=function(polygons,point){var ncross=0;for(var i=0;i<polygons.length;i++){var polygon=polygons[i];polygon.push(polygon[0]);for(var j=0;j<polygon.length-1;j++){var point1=polygon[j];var point2=polygon[j+1];if(point1[1]>=point[1]&&point2[1]<point[1]||point1[1]<=point[1]&&point2[1]>point[1]){var lon=point1[0]+(point[1]-point1[1])*(point2[0]-point1[0])/(point2[1]-point1[1]);if(point[0]>lon){ncross+=1}}}}return ncross%2===1};var squareInfield=function(border,ne,sw){var northSouthout=ne[0]<border._southWest.lat;var southNorthout=border._northEast.lat<sw[0];var eastWestout=ne[1]<border._southWest.lng;var westEastout=border._northEast.lng<sw[1];return!(northSouthout||southNorthout||eastWestout||westEastout)};var identify=function(latlon){var city="";var parent=[];var citykey=[];Map.setView(latlon,18);var border=Map.getBounds();for(var key in relm){if(squareInfield(border,relm[key].ne,relm[key].sw)){city+=(city?",":"")+relm[key].name;parent.push(relm[key].parent);citykey.push(key)}}var identify=checkInclusion(latlon,citykey);return identify[0]}})(typeof window==="object"?window:typeof global==="object"?global:this); </script> 【略】 <script> (function(root){"use strict";var Const={};Const.TELOPS={100:["100.svg","500.svg","100","晴","CLEAR"],101:["101.svg","501.svg","100","晴時々曇","PARTLY CLOUDY"],102:["102.svg","502.svg","300","晴一時雨","CLEAR, OCCASIONAL SCATTERED SHOWERS"],103:["102.svg","502.svg","300","晴時々雨","CLEAR, FREQUENT SCATTERED SHOWERS"],104:["104.svg","504.svg","400","晴一時雪","CLEAR, SNOW FLURRIES"],105:["104.svg","504.svg","400","晴時々雪","CLEAR, FREQUENT SNOW FLURRIES"],106:["102.svg","502.svg","300","晴一時雨か雪","CLEAR, OCCASIONAL SCATTERED SHOWERS OR SNOW FLURRIES"],107:["102.svg","502.svg","300","晴時々雨か雪","CLEAR, FREQUENT SCATTERED SHOWERS OR SNOW FLURRIES"],108:["102.svg","502.svg","300","晴一時雨か雷 雨","CLEAR, OCCASIONAL SCATTERED SHOWERS AND/OR THUNDER"],110:["110.svg","510.svg","100","晴後時々曇","CLEAR, PARTLY CLOUDY LATER"],111:["110.svg","510.svg","100","晴後曇","CLEAR, CLOUDY LATER"],112:["112.svg","512.svg","300","晴後一時 雨","CLEAR, OCCASIONAL SCATTERED SHOWERS LATER"],113:["112.svg","512.svg","300","晴後時々雨","CLEAR, FREQUENT SCATTERED SHOWERS LATER"],114:["112.svg","512.svg","300","晴後雨","CLEAR,RAIN LATER"],115:["115.svg","515.svg","400","晴後一時雪","CLEAR, OCCASIONAL SNOW FLURRIES LATER"],116:["115.svg","515.svg","400","晴後時々雪","CLEAR, FREQUENT SNOW FLURRIES LATER"],117:["115.svg","515.svg","400","晴後雪","CLEAR,SNOW LATER"],118:["112.svg","512.svg","300","晴後雨か雪","CLEAR, RAIN OR SNOW LATER"],119:["112.svg","512.svg","300","晴後雨か雷雨","CLEAR, RAIN AND/OR THUNDER LATER"],120:["102.svg","502.svg","300","晴朝夕一時雨","OCCASIONAL SCATTERED SHOWERS IN THE MORNING AND EVENING, CLEAR DURING THE DAY"],121:["102.svg","502.svg","300","晴朝の内一時雨","OCCASIONAL SCATTERED SHOWERS IN THE MORNING, CLEAR DURING THE DAY"],122:["112.svg","512.svg","300","晴夕方一時雨","CLEAR, OCCASIONAL SCATTERED SHOWERS IN THE EVENING"],123:["100.svg","500.svg","100","晴山沿い雷雨","CLEAR IN THE PLAINS, RAIN AND THUNDER NEAR MOUTAINOUS AREAS"],124:["100.svg","500.svg","100","晴山沿い雪","CLEAR IN THE PLAINS, SNOW NEAR MOUTAINOUS AREAS"],125:["112.svg","512.svg","300","晴午後は雷雨","CLEAR, RAIN AND THUNDER IN THE AFTERNOON"],126:["112.svg","512.svg","300","晴昼頃から雨","CLEAR, RAIN IN THE AFTERNOON"],127:["112.svg","512.svg","300","晴夕方から雨","CLEAR, RAIN IN THE EVENING"],128:["112.svg","512.svg","300","晴夜は雨","CLEAR, RAIN IN THE NIGHT"],130:["100.svg","500.svg","100","朝の内霧後晴","FOG IN THE MORNING, CLEAR LATER"],131:["100.svg","500.svg","100","晴明け方霧","FOG AROUND DAWN, CLEAR LATER"],132:["101.svg","501.svg","100","晴朝夕曇","CLOUDY IN THE MORNING AND EVENING, CLEAR DURING THE DAY"],140:["102.svg","502.svg","300","晴時々雨で雷を伴う","CLEAR, FREQUENT SCATTERED SHOWERS AND THUNDER"],160:["104.svg","504.svg","400","晴一時雪か雨","CLEAR, SNOW FLURRIES OR OCCASIONAL SCATTERED SHOWERS"],170:["104.svg","504.svg","400","晴時々雪か雨","CLEAR, FREQUENT SNOW FLURRIES OR SCATTERED SHOWERS"],181:["115.svg","515.svg","400","晴後雪か雨","CLEAR, SNOW OR RAIN LATER"],200:["200.svg","200.svg","200","曇","CLOUDY"],201:["201.svg","601.svg","200","曇時々晴","MOSTLY CLOUDY"],202:["202.svg","202.svg","300","曇一時雨","CLOUDY, OCCASIONAL SCATTERED SHOWERS"],203:["202.svg","202.svg","300","曇時々雨","CLOUDY, FREQUENT SCATTERED SHOWERS"],204:["204.svg","204.svg","400","曇一時雪","CLOUDY, OCCASIONAL SNOW FLURRIES"],205:["204.svg","204.svg","400","曇時々雪","CLOUDY FREQUENT SNOW FLURRIES"],206:["202.svg","202.svg","300","曇一時雨か雪","CLOUDY, OCCASIONAL SCATTERED SHOWERS OR SNOW FLURRIES"],207:["202.svg","202.svg","300","曇時々雨か雪","CLOUDY, FREQUENT SCCATERED SHOWERS OR SNOW FLURRIES"],208:["202.svg","202.svg","300","曇一時雨か雷雨","CLOUDY, OCCASIONAL SCATTERED SHOWERS AND/OR THUNDER"],209:["200.svg","200.svg","200","霧","FOG"],210:["210.svg","610.svg","200","曇後時々晴","CLOUDY, PARTLY CLOUDY LATER"],211:["210.svg","610.svg","200","曇後晴","CLOUDY, CLEAR LATER"],212:["212.svg","212.svg","300","曇後一時雨","CLOUDY, OCCASIONAL SCATTERED SHOWERS LATER"],213:["212.svg","212.svg","300","曇後時々雨","CLOUDY, FREQUENT SCATTERED SHOWERS LATER"],214:["212.svg","212.svg","300","曇後雨","CLOUDY, RAIN LATER"],215:["215.svg","215.svg","400","曇後一時雪","CLOUDY, SNOW FLURRIES LATER"],216:["215.svg","215.svg","400","曇後時々雪","CLOUDY, FREQUENT SNOW FLURRIES LATER"],217:["215.svg","215.svg","400","曇後雪","CLOUDY, SNOW LATER"],218:["212.svg","212.svg","300","曇後雨か雪","CLOUDY, RAIN OR SNOW LATER"],219:["212.svg","212.svg","300","曇後雨か雷雨","CLOUDY, RAIN AND/OR THUNDER LATER"],220:["202.svg","202.svg","300","曇朝夕一時雨","OCCASIONAL SCCATERED SHOWERS IN THE MORNING AND EVENING, CLOUDY DURING THE DAY"],221:["202.svg","202.svg","300","曇朝の内一時雨","CLOUDY OCCASIONAL SCCATERED SHOWERS IN THE MORNING"],222:["212.svg","212.svg","300","曇夕方一時雨","CLOUDY, OCCASIONAL SCCATERED SHOWERS IN THE EVENING"],223:["201.svg","601.svg","200","曇日中時々晴","CLOUDY IN THE MORNING AND EVENING, PARTLY CLOUDY DURING THE DAY,"],224:["212.svg","212.svg","300","曇昼頃から雨","CLOUDY, RAIN IN THE AFTERNOON"],225:["212.svg","212.svg","300","曇夕方から雨","CLOUDY, RAIN IN THE EVENING"],226:["212.svg","212.svg","300","曇夜は雨","CLOUDY, RAIN IN THE NIGHT"],228:["215.svg","215.svg","400","曇昼頃から雪","CLOUDY, SNOW IN THE AFTERNOON"],229:["215.svg","215.svg","400","曇夕方から雪","CLOUDY, SNOW IN THE EVENING"],230:["215.svg","215.svg","400","曇夜は雪","CLOUDY, SNOW IN THE NIGHT"],231:["200.svg","200.svg","200","曇海上海岸は霧か霧雨","CLOUDY, FOG OR DRIZZLING ON THE SEA AND NEAR SEASHORE"],240:["202.svg","202.svg","300","曇時々雨で雷を伴う","CLOUDY, FREQUENT SCCATERED SHOWERS AND THUNDER"],250:["204.svg","204.svg","400","曇時々雪で雷を伴う","CLOUDY, FREQUENT SNOW AND THUNDER"],260:["204.svg","204.svg","400","曇一時雪か雨","CLOUDY, SNOW FLURRIES OR OCCASIONAL SCATTERED SHOWERS"],270:["204.svg","204.svg","400","曇時々雪か雨","CLOUDY, FREQUENT SNOW FLURRIES OR SCATTERED SHOWERS"],281:["215.svg","215.svg","400","曇後雪か雨","CLOUDY, SNOW OR RAIN LATER"],300:["300.svg","300.svg","300","雨","RAIN"],301:["301.svg","701.svg","300","雨時々晴","RAIN, PARTLY CLOUDY"],302:["302.svg","302.svg","300","雨時々止む","SHOWERS THROUGHOUT THE DAY"],303:["303.svg","303.svg","400","雨時々雪","RAIN,FREQUENT SNOW FLURRIES"],304:["300.svg","300.svg","300","雨か雪","RAINORSNOW"],306:["300.svg","300.svg","300","大雨","HEAVYRAIN"],308:["308.svg","308.svg","300","雨で暴風を伴う","RAINSTORM"],309:["303.svg","303.svg","400","雨一時雪","RAIN,OCCASIONAL SNOW"],311:["311.svg","711.svg","300","雨後晴","RAIN,CLEAR LATER"],313:["313.svg","313.svg","300","雨後曇","RAIN,CLOUDY LATER"],314:["314.svg","314.svg","400","雨後時々雪","RAIN, FREQUENT SNOW FLURRIES LATER"],315:["314.svg","314.svg","400","雨後雪","RAIN,SNOW LATER"],316:["311.svg","711.svg","300","雨か雪後晴","RAIN OR SNOW, CLEAR LATER"],317:["313.svg","313.svg","300","雨か雪後曇","RAIN OR SNOW, CLOUDY LATER"],320:["311.svg","711.svg","300","朝の内雨 後晴","RAIN IN THE MORNING, CLEAR LATER"],321:["313.svg","313.svg","300","朝の内雨後曇","RAIN IN THE MORNING, CLOUDY LATER"],322:["303.svg","303.svg","400","雨朝晩一時雪","OCCASIONAL SNOW IN THE MORNING AND EVENING, RAIN DURING THE DAY"],323:["311.svg","711.svg","300","雨昼頃から晴","RAIN, CLEAR IN THE AFTERNOON"],324:["311.svg","711.svg","300","雨夕方から晴","RAIN, CLEAR IN THE EVENING"],325:["311.svg","711.svg","300","雨夜は晴","RAIN, CLEAR IN THE NIGHT"],326:["314.svg","314.svg","400","雨夕方から雪","RAIN, SNOW IN THE EVENING"],327:["314.svg","314.svg","400","雨夜は雪","RAIN,SNOW IN THE NIGHT"],328:["300.svg","300.svg","300","雨一時強く降る","RAIN, EXPECT OCCASIONAL HEAVY RAINFALL"],329:["300.svg","300.svg","300","雨一時みぞれ","RAIN, OCCASIONAL SLEET"],340:["400.svg","400.svg","400","雪か雨","SNOWORRAIN"],350:["300.svg","300.svg","300","雨で雷を伴う","RAIN AND THUNDER"],361:["411.svg","811.svg","400","雪か雨後晴","SNOW OR RAIN, CLEAR LATER"],371:["413.svg","413.svg","400","雪か雨後曇","SNOW OR RAIN, CLOUDY LATER"],400:["400.svg","400.svg","400","雪","SNOW"],401:["401.svg","801.svg","400","雪時々晴","SNOW, FREQUENT CLEAR"],402:["402.svg","402.svg","400","雪時々止む","SNOWTHROUGHOUT THE DAY"],403:["403.svg","403.svg","400","雪時々雨","SNOW,FREQUENT SCCATERED SHOWERS"],405:["400.svg","400.svg","400","大雪","HEAVYSNOW"],406:["406.svg","406.svg","400","風雪強い","SNOWSTORM"],407:["406.svg","406.svg","400","暴風雪","HEAVYSNOWSTORM"],409:["403.svg","403.svg","400","雪一時雨","SNOW, OCCASIONAL SCCATERED SHOWERS"],411:["411.svg","811.svg","400","雪後晴","SNOW,CLEAR LATER"],413:["413.svg","413.svg","400","雪後曇","SNOW,CLOUDY LATER"],414:["414.svg","414.svg","400","雪後雨","SNOW,RAIN LATER"],420:["411.svg","811.svg","400","朝の内雪後晴","SNOW IN THE MORNING, CLEAR LATER"],421:["413.svg","413.svg","400","朝の内雪後曇","SNOW IN THE MORNING, CLOUDY LATER"],422:["414.svg","414.svg","400","雪昼頃から雨","SNOW, RAIN IN THE AFTERNOON"],423:["414.svg","414.svg","400","雪夕方から雨","SNOW, RAIN IN THE EVENING"],425:["400.svg","400.svg","400","雪一時強く降る","SNOW, EXPECT OCCASIONAL HEAVY SNOWFALL"],426:["400.svg","400.svg","400","雪後みぞれ","SNOW, SLEET LATER"],427:["400.svg","400.svg","400","雪一時みぞれ","SNOW, OCCASIONAL SLEET"],450:["400.svg","400.svg","400","雪で雷を伴う","SNOW AND THUNDER"]};Const.AREA_FUKEN=[{center:"016000",offices:["016000","011000","013000","014030","014100","015000","012000","017000"]},{center:"040000",offices:["040000","060000","070000","020000","050000","030000"]},{center:"130000",offices:["130000","120000","140000","190000","090000","100000","110000","080000","200000"]},{center:"150000",offices:["150000","170000","180000","160000"]},{center:"230000",offices:["230000","240000","220000","210000"]},{center:"270000",offices:["270000","300000","260000","250000","280000","290000"]},{center:"340000",offices:["340000","310000","330000","320000"]},{center:"370000",offices:["370000","380000","360000","390000"]},{center:"400000",offices:["400000","440000","410000","430000","420000","350000"]},{center:"460100",offices:["460100","450000","460040"]},{center:"471000",offices:["471000","473000","474000","472000"]}];Const.WEEK_AREAS={"011000":"Soya Region","012000":"Kamikawa Rumoi Region","013000":"Abashiri Kitami Mombetsu Region","014000":"Kushiro Nemuro Tokachi Region","014030":"Tokachi Region","014100":"Kushiro Nemuro Region","015000":"Iburi Hidaka Region","016000":"Ishikari Sorachi Shiribeshi Region","017000":"Oshima Hiyama Region","020000":"Aomori Prefecture","020010":"Tsugaru","020030":"Sanpachi Kamikita","020100":"Tsugaru Shimokita","020200":"Shimokita Sanpachi Kamikita","030000":"Iwate Prefecture","030010":"Inland","030100":"Coast","040000":"Miyagi Prefecture","040010":"Eastern Region","040020":"Western Region","050000":"Akita Prefecture","060000":"Yamagata Prefecture","070000":"Fukushima Prefecture","070030":"Aizu","070100":"Nakadori Hamadori","080000":"Ibaraki Prefecture","090000":"Tochigi Prefecture",1e5:"Gunma Prefecture",100010:"Southern Region",100020:"Northern Region",11e4:"Saitama Prefecture",12e4:"Chiba Prefecture",130010:"Tokyo Region",130020:"Northern Izu Islands",130030:"Southern Izu Islands",130040:"Ogasawara Islands",130100:"Izu Islands",14e4:"Kanagawa Prefecture",15e4:"Niigata Prefecture",16e4:"Toyama Prefecture",17e4:"Ishikawa Prefecture",18e4:"Fukui Prefecture",19e4:"Yamanashi Prefecture",2e5:"Nagano Prefecture",200010:"Northern Region",200100:"Central Region Southern Region",21e4:"Gifu Prefecture",210010:"Mino Region",210020:"Hida Region",22e4:"Shizuoka Prefecture",23e4:"Aichi Prefecture",24e4:"Mie Prefecture",25e4:"Shiga Prefecture",250010:"Southern Region",250020:"Northern Region",26e4:"Kyoto Prefecture",260010:"Southern Region",260020:"Northern Region",27e4:"Osaka Prefecture",28e4:"Hyogo Prefecture",280010:"Southern Region",280020:"Northern Region",29e4:"Nara Prefecture",3e5:"Wakayama Prefecture",31e4:"Tottori Prefecture",32e4:"Shimane Prefecture",33e4:"Okayama Prefecture",330010:"Southern Region",330020:"Northern Region",34e4:"Hiroshima Prefecture",340010:"Southern Region",340020:"Northern Region",35e4:"Yamaguchi Prefecture",36e4:"Tokushima Prefecture",37e4:"Kagawa Prefecture",38e4:"Ehime Prefecture",39e4:"Kochi Prefecture",4e5:"Fukuoka Prefecture",41e4:"Saga Prefecture",42e4:"Nagasaki Prefecture",420030:"Iki Tsushima",420100:"Southern Region<br>Northern Region<br>Goto",43e4:"Kumamoto Prefecture",44e4:"Oita Prefecture",45e4:"Miyazaki Prefecture",460040:"Amami Region",460100:"Kagoshima Prefecture",471e3:"Okinawa Main Island Region",472e3:"Daitojima Region",473e3:"Miyakojima Region",474e3:"Yaeyama Region"};root.Forecast=root.Forecast||{};root.Forecast.Const=Const})(typeof window==="object"?window:typeof global==="object"?global:this); </script> 【略】 <script> (function(root){"use strict";var util=JMAWebUtility;var hashC=util.hashController;var TR=Forecast.translator;var makeFweekURL=function(areas,areaType,areaCode,basePath){var code,pathCode;if(areaType==="japan"){code=areaCode}else{code=util.area.convertCode(areas,areaCode,areaType,"offices")}pathCode=Forecast.Common.getPathCode(code);return basePath+"data/forecast/"+pathCode+".json"};var insertContents=function(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,explain,areaType,areaCode,basePath){var weekAreaButtons=$("#forecast-area-buttons").empty();var shortTableContainer=$("#short-table-container").empty();var shortOverviewContainer=$("#short-overview-container").empty();var shortExplainContainer=$("#short-explain-container").empty();var weekTableContainer=$("#week-table-container").empty();var weekOverviewContainer=$("#week-overview-container").empty();var weekExplainContainer=$("#week-explain-container").empty();var fweekURL=makeFweekURL(areas,areaType,areaCode,basePath);var isDayTime;util.parallelRequest({fweek:fweekURL},function(data){var fweek=data.fweek;var hour=util.date.getHours(new Date(fweek[0].reportDatetime));ContentsCommon.makeBreadCrumbs(areaType,areaCode,TR.tr("天気予報"),areas).appendTo(weekAreaButtons);if(areaType!=="japan"){Forecast.Short.Table.insert(shortTableContainer,areas,shortAreas,anniversary,amedasTable,fweek,areaType,areaCode,basePath);if(TR.isJapanese()){Forecast.Overview.insert(shortOverviewContainer,areas,areaType,areaCode,"short")}if(hour){isDayTime=hour>=11||hour<5;ContentsCommon.makeExplain(shortExplainContainer,$(explain).filter("div.forecast-explain-short"+TR.select("","-en")),{title:TR.select("天気予報("+(isDayTime?"明後日":"明日")+"までの詳細)の説明","")})}}Forecast.Week.Table.insert(weekTableContainer,areas,weekArea,weekArea05,anniversary,amedasTable,fweek,areaType,areaCode,basePath,false);if(TR.isJapanese()){Forecast.Overview.insert(weekOverviewContainer,areas,areaType,areaCode,"week")}if(hour){isDayTime=hour>=11||hour<5;ContentsCommon.makeExplain(weekExplainContainer,areaType==="japan"?$(explain).filter("div.forecast-explain-week-japan"+TR.select("","-en")):$(explain).filter("div.forecast-explain-week"+TR.select("","-en")),{title:TR.select("天気予報 ("+(isDayTime?"7":"6")+"日先まで)の説明","")})}})};var initialize=function(){var basePath="./";util.parallelRequest({areas:"../common/const/area.json",contents:"../common/const/contents.json",weekArea:"const/week_area.json",anniversary:"const/anniversary.json",weekArea05:"const/week_area05.json",shortArea:"const/forecast_area.json",amedasTable:"../amedas/const/amedastable.json",explain:"const/explain.html"},function(data){var areas=data.areas;var contents=data.contents;var hash=fixHash(hashC.getHashData(),areas);var areaType=hash.area_type;var areaCode=hash.area_code;var weekArea=data.weekArea;var anniversary=data.anniversary;var weekArea05=data.weekArea05;var shortAreas=data.shortArea;var amedasTable=data.amedasTable;var areaMenu;areaMenu=insertAreaMenu(areas,contents,areaType,areaCode);insertContents(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,data.explain,areaType,areaCode,basePath);setEvent(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,data.explain,areaMenu,basePath)})};var fixHash=function(originalHash,areas){var hash={};var areaType=originalHash["area_type"];var areaCode=originalHash["area_code"];if((areaType==="offices"||areaType==="class20s")&&areas[areaType][areaCode]){hash["area_type"]=areaType;hash["area_code"]=areaCode}else if((areaType==="class10s"||areaType==="class15s")&&areas[areaType][areaCode]){hash["area_type"]="offices";hash["area_code"]=util.area.convertCode(areas,areaCode,areaType,"offices")}else{hash["area_type"]="japan";hash["area_code"]="010000"}return hash};var insertAreaMenu=function(areas,contents,areaType,areaCode){var areaMenuMaker=new AreaMenuMaker(areas);return areaMenuMaker.make(areaType,areaCode,TR.tr("天気予報"),"header",{map_url:"../map.html#contents=forecast",contents:contents,current_content:"forecast"})};var setEvent=function(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,explain,areaMenu,basePath){window.addEventListener("hashchange",function(){var hash=fixHash(hashC.getHashData(),areas);var areaType=hash["area_type"];var areaCode=hash["area_code"];areaMenu.setValue(hash);insertContents(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,explain,areaType,areaCode,basePath)})};var insert=function(tag,areaType,areaCode,_officeCode,options){util.parallelRequest({weekArea:"forecast/const/week_area.json",anniversary:"forecast/const/anniversary.json",weekArea05:"forecast/const/week_area05.json",amedasTable:"amedas/const/amedastable.json"},function(data){var weekTableContainer=$(tag);var areas=options.areas;var weekArea=data.weekArea;var anniversary=data.anniversary;var weekArea05=data.weekArea05;var amedasTable=data.amedasTable;var basePath="forecast/";var shortened=true;var fweekURL=makeFweekURL(areas,areaType,areaCode,basePath);util.parallelRequest({fweek:fweekURL},function(data2){var fweek=data2.fweek;Forecast.Week.Table.insert(weekTableContainer,areas,weekArea,weekArea05,anniversary,amedasTable,fweek,areaType,areaCode,basePath,shortened)})})};root.Forecast=root.Forecast||{};root.Forecast.initialize=initialize;root.Forecast.insert=insert})(typeof window==="object"?window:typeof global==="object"?global:this); </script> <script> $(function () { Forecast.initialize(); }); </script> </body> </html> |
[スポンサーリンク]
Requests-HTMLライブラリで動的ページをスクレイピング
Requests-HTMLライブラリとは
PythonでHTMLをパースすることが出来るライブラリです
requestsライブラリとの大きな違いの1つが、JavaScriptを完全サポートしていることです
このライブラリを使用することで、JavaScriptが生成したHTMLコードを取得することが出来ます
また、Beautiful Soupライブラリと同様にパースも行うことが出来ます
Requests-HTMLライブラリのインストール
Requests-HTMLライブラリはpipコマンドでインストールすることが出来ます
1 |
pip install requests-html |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
# pip install requests-html WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead. Collecting requests-html Downloading https://files.pythonhosted.org/packages/24/bc/a4380f09bab3a776182578ce6b2771e57259d0d4dbce178205779abdc347/requests_html-0.10.0-py3-none-any.whl Requirement already satisfied: requests in /usr/local/lib/python3.6/site-packages (from requests-html) Collecting fake-useragent (from requests-html) Downloading https://files.pythonhosted.org/packages/d1/79/af647635d6968e2deb57a208d309f6069d31cb138066d7e821e575112a80/fake-useragent-0.1.11.tar.gz Collecting parse (from requests-html) Downloading https://files.pythonhosted.org/packages/89/a1/82ce536be577ba09d4dcee45db58423a180873ad38a2d014d26ab7b7cb8a/parse-1.19.0.tar.gz Collecting pyppeteer>=0.0.14 (from requests-html) Downloading https://files.pythonhosted.org/packages/4a/05/ea3250282e46fda60df1f1d5246bb8cdc022abb89969c61a98ea28fd6b82/pyppeteer-0.2.5-py3-none-any.whl (87kB) 100% |????????????????????????????????| 92kB 4.0MB/s Collecting w3lib (from requests-html) Downloading https://files.pythonhosted.org/packages/a3/59/b6b14521090e7f42669cafdb84b0ab89301a42f1f1a82fcf5856661ea3a7/w3lib-1.22.0-py2.py3-none-any.whl Collecting pyquery (from requests-html) Downloading https://files.pythonhosted.org/packages/58/0b/85d15e21f660a8ea68b1e0286168938857391f4ec9f6d204d91c9e013826/pyquery-1.4.3-py3-none-any.whl Collecting bs4 (from requests-html) Downloading https://files.pythonhosted.org/packages/10/ed/7e8b97591f6f456174139ec089c769f89a94a1a4025fe967691de971f314/bs4-0.0.1.tar.gz Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/site-packages (from requests->requests-html) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/site-packages (from requests->requests-html) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/site-packages (from requests->requests-html) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests->requests-html) Collecting appdirs<2.0.0,>=1.4.3 (from pyppeteer>=0.0.14->requests-html) Downloading https://files.pythonhosted.org/packages/3b/00/2344469e2084fb287c2e0b57b72910309874c3245463acd6cf5e3db69324/appdirs-1.4.4-py2.py3-none-any.whl Collecting websockets<9.0,>=8.1 (from pyppeteer>=0.0.14->requests-html) Downloading https://files.pythonhosted.org/packages/cf/cb/c35513c4a0ff24ca13e33f7336ba8c1a864449fad9fea8e37abdad11c38d/websockets-8.1-cp36-cp36m-manylinux1_x86_64.whl (73kB) 100% |????????????????????????????????| 81kB 7.8MB/s Collecting pyee<9.0.0,>=8.1.0 (from pyppeteer>=0.0.14->requests-html) Downloading https://files.pythonhosted.org/packages/0d/0a/933b3931107e1da186963fd9bb9bceb9a613cff034cb0fb3b0c61003f357/pyee-8.1.0-py2.py3-none-any.whl Collecting importlib-metadata<3.0.0,>=2.1.1; python_version < "3.8" (from pyppeteer>=0.0.14->requests-html) Downloading https://files.pythonhosted.org/packages/98/b8/8ec57a8ef46fbe7f185318c7ff7df9a06c9df451d9a59a067bfa851bb828/importlib_metadata-2.1.1-py2.py3-none-any.whl Collecting tqdm<5.0.0,>=4.42.1 (from pyppeteer>=0.0.14->requests-html) Downloading https://files.pythonhosted.org/packages/4e/8c/f1035bd24b0e352ddba7be320abc1603fc4c9976fcda6971ed287be59164/tqdm-4.58.0-py2.py3-none-any.whl (73kB) 100% |????????????????????????????????| 81kB 7.7MB/s Collecting six>=1.4.1 (from w3lib->requests-html) Downloading https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl Collecting lxml>=2.1 (from pyquery->requests-html) Downloading https://files.pythonhosted.org/packages/bd/78/56a7c88a57d0d14945472535d0df9fb4bbad7d34ede658ec7961635c790e/lxml-4.6.2-cp36-cp36m-manylinux1_x86_64.whl (5.5MB) 100% |????????????????????????????????| 5.5MB 276kB/s Collecting cssselect>0.7.9 (from pyquery->requests-html) Downloading https://files.pythonhosted.org/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.6/site-packages (from bs4->requests-html) Collecting zipp>=0.5 (from importlib-metadata<3.0.0,>=2.1.1; python_version < "3.8"->pyppeteer>=0.0.14->requests-html) Downloading https://files.pythonhosted.org/packages/41/ad/6a4f1a124b325618a7fb758b885b68ff7b058eec47d9220a12ab38d90b1f/zipp-3.4.0-py3-none-any.whl Requirement already satisfied: soupsieve>=1.2 in /usr/local/lib/python3.6/site-packages (from beautifulsoup4->bs4->requests-html) Installing collected packages: fake-useragent, parse, appdirs, websockets, pyee, zipp, importlib-metadata, tqdm, pyppeteer, six, w3lib, lxml, cssselect, pyquery, bs4, requests-html Running setup.py install for fake-useragent ... done Running setup.py install for parse ... done Running setup.py install for bs4 ... done Successfully installed appdirs-1.4.4 bs4-0.0.1 cssselect-1.1.0 fake-useragent-0.1.11 importlib-metadata-2.1.1 lxml-4.6.2 parse-1.19.0 pyee-8.1.0 pyppeteer-0.2.5 pyquery-1.4.3 requests-html-0.10.0 six-1.15.0 tqdm-4.58.0 w3lib-1.22.0 websockets-8.1 zipp-3.4.0 # |
インストールした時点では0.10.0がインストールされました
1 |
pip show requests-html |
1 2 3 4 5 6 7 8 9 10 11 |
# pip show requests-html Name: requests-html Version: 0.10.0 Summary: HTML Parsing for Humans. Home-page: https://github.com/kennethreitz/requests-html Author: Kenneth Reitz Author-email: me@kennethreitz.org License: MIT Location: /usr/local/lib/python3.6/site-packages Requires: w3lib, fake-useragent, pyquery, bs4, parse, requests, pyppeteer # |
pyppeteerライブラリのインストール
次にpyppeteerライブラリをインストールします
pyppeteerライブラリはChromeのヘッドレスモードを操作するライブラリです
ヘッドレスブラウザと呼ばれるもので、よく使っているEdgeやChromeのアプリでの画面表示ではことなり描写は行いません
JavaScriptのテスト等の為に、グラフィック描写をせずにScriptの実行などのみを行う為に使用します
Requests-HTMLライブラリを初回に実行した際に、自動でインストールされますが、今回は事前にインストールを行います
Requests-HTMLライブラリと同様にpipコマンドでインストール出来ます
1 |
pip install pyppeteer |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# pip install pyppeteer WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead. Requirement already satisfied: pyppeteer in /usr/local/lib/python3.6/site-packages Collecting urllib3<2.0.0,>=1.25.8 (from pyppeteer) Downloading https://files.pythonhosted.org/packages/23/fc/8a49991f7905261f9ca9df5aa9b58363c3c821ce3e7f671895442b7100f2/urllib3-1.26.3-py2.py3-none-any.whl (137kB) 100% |????????????????????????????????| 143kB 3.5MB/s Requirement already satisfied: importlib-metadata<3.0.0,>=2.1.1; python_version < "3.8" in /usr/local/lib/python3.6/site-packages (from pyppeteer) Requirement already satisfied: pyee<9.0.0,>=8.1.0 in /usr/local/lib/python3.6/site-packages (from pyppeteer) Requirement already satisfied: websockets<9.0,>=8.1 in /usr/local/lib64/python3.6/site-packages (from pyppeteer) Requirement already satisfied: tqdm<5.0.0,>=4.42.1 in /usr/local/lib/python3.6/site-packages (from pyppeteer) Requirement already satisfied: appdirs<2.0.0,>=1.4.3 in /usr/local/lib/python3.6/site-packages (from pyppeteer) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/site-packages (from importlib-metadata<3.0.0,>=2.1.1; python_version < "3.8"->pyppeteer) Installing collected packages: urllib3 Found existing installation: urllib3 1.25.7 Uninstalling urllib3-1.25.7: Successfully uninstalled urllib3-1.25.7 Successfully installed urllib3-1.26.3 # |
<class ‘pyppeteer.errors.browsererror’> Browser closed unexpectedly:
この後に実際にスクレイピングした際に、pyppeteer.errors.BrowserErrorが発生しました
調べていると下段のページに辿り着きました(非常に助かる情報でした)
pip install pyppeteerだけでは必要なライブラリが全てインストールされないようです
こちらの情報を元に、必要と記載されているライブラリをインストールするとエラーが発生しなくなりました
ヘッドレストモードで起動する場合でも、X Window System等が必要なのかもしれません
1 |
yum -y install libX11 libXcomposite libXcursor libXdamage libXext libXi libXtst cups-libs libXScrnSaver libXrandr alsa-lib pango atk at-spi2-atk gtk3 |
[スポンサーリンク]
Requests-HTMLライブラリで動的ページの表示されるHTMLを取得
Requests-HTMLライブラリを使用して動的ページの表示されているHTMLを取得していきます
Requests-HTMLライブラリ 動的ページHTML取得 サンプルコード
・HTMLを取得するURLを設定
1 |
ScrapingGetHtmlUrl = 【Webスクレイピング対象URL】 |
・Requests-HTMLのセッションを作成しソースコードを取得
1 2 |
ScrapingSession = HTMLSession() ScrapingGetHtmlResponse = ScrapingSession.get(ScrapingGetHtmlUrl) |
・JavaScriptをレンダリング
render()でレンダリングをしてJavaScriptの実行結果を取得
asyncの場合は、arender()を使用
※timeout、sleepは後述
1 |
ScrapingGetHtmlResponse.html.render() |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
# coding: utf_8 # モジュールインポート import requests # WEBスクレイピングパラメータ設定 ScrapingGetHtmlUrl = 【Webスクレイピング対象URL】 # WEBスクレイピング実行 ScrapingSession = HTMLSession() ScrapingGetHtmlResponse = ScrapingSession.get(ScrapingGetHtmlUrl) ScrapingGetHtmlResponse.html.render() if ScrapingGetHtmlResponse.status_code == 200: print(ScrapingGetHtmlResponse.html.html) else: print("WEBスクレイピングに失敗") print(ScrapingGetHtmlResponse) |
Requests-HTMLライブラリを使用してもJavaScriptの実行結果が取得出来ない場合
サンプルコードの【Webスクレイピング対象URL】に、気象庁様の大阪府の天気予報のサイトをして実行しました
しかし、取得されたのはrequestsライブラリと同じくJavaScritptが含まれたコードでした
こちらは、下記のサイト様の情報を基に、render()にtimeoutとsleepを設定することで解消することが出来ました
レンダリングが未完了でも後続処理が進むようです
timeout=0(無制限)、sleep=10を設定することでレンダリング後のHTMLを取得出来ました
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
# coding: utf_8 # モジュールインポート import requests # WEBスクレイピングパラメータ設定 ScrapingGetHtmlUrl = 'https://www.jma.go.jp/bosai/forecast/#area_type=offices&area_code=270000' # WEBスクレイピング実行 ScrapingSession = HTMLSession() ScrapingGetHtmlResponse = ScrapingSession.get(ScrapingGetHtmlUrl) ScrapingGetHtmlResponse.html.render(timeout=0,sleep=10) if ScrapingGetHtmlResponse.status_code == 200: print(ScrapingGetHtmlResponse.html.html) else: print("WEBスクレイピングに失敗") print(ScrapingGetHtmlResponse) |
「くもり 時々 晴れ」のように、JavaScriptの実行結果が取得出来ていることが分かります
これでJavaScriptを使用した動的なページも、実行結果として生成されたHTMLが取得することが出来ます
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
<!DOCTYPE html><html lang="ja"><head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width,initial-scale=1"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>気象庁|天気予報</title> <style> @charset "UTF-8";/*! * Bootstrap Reboot v4.1.3 (https://getbootstrap.com/) * Copyright 2011-2018 The Bootstrap Authors * Copyright 2011-2018 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}html{font-family:sans-serif;line-height:1.15;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%;-ms-overflow-style:scrollbar;-webkit-tap-highlight-color:transparent}@-ms-viewport{width:device-width}article,aside,figcaption,figure,footer,header,hgroup,main,nav,section{display:block}body{margin:0;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;text-align:left;background-color:#fff}[tabindex="-1"]:focus{outline:0!important}hr{box-sizing:content-box;height:0;overflow:visible}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem}p{margin-top:0;margin-bottom:1rem}abbr[data-original-title],abbr[title]{text-decoration:underline;-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;border-bottom:0}address{margin-bottom:1rem;font-style:normal;line-height:inherit}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}dfn{font-style:italic}b,strong{font-weight:bolder}small{font-size:80%}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#007bff;text-decoration:none;background-color:transparent;-webkit-text-decoration-skip:objects}a:hover{color:#0056b3;text-decoration:underline}a:not([href]):not([tabindex]){text-decoration:none}a:not([href]):not([tabindex]):focus,a:not([href]):not([tabindex]):hover{text-decoration:none}a:not([href]):not([tabindex]):focus{outline:0}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em}pre{margin-top:0;margin-bottom:1rem;overflow:auto;-ms-overflow-style:scrollbar}figure{margin:0 0 1rem}img{vertical-align:middle;border-style:none;box-sizing:content-box}svg{overflow:hidden;vertical-align:middle}table{border-collapse:collapse}caption{padding-top:.75rem;padding-bottom:.75rem;color:#6c757d;text-align:left;caption-side:bottom}th{text-align:inherit}label{display:inline-block;margin-bottom:.5rem}button{border-radius:0}button:focus{outline:1px dotted;outline:5px auto -webkit-focus-ring-color}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,input{overflow:visible}button,select{text-transform:none}[type=reset],[type=submit],button,html [type=button]{-webkit-appearance:button}[type=button]::-moz-focus-inner,[type=reset]::-moz-focus-inner,[type=submit]::-moz-focus-inner,button::-moz-focus-inner{padding:0;border-style:none}input[type=checkbox],input[type=radio]{box-sizing:border-box;padding:0}input[type=date],input[type=datetime-local],input[type=month],input[type=time]{-webkit-appearance:listbox}textarea{overflow:auto;resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{display:block;width:100%;max-width:100%;padding:0;margin-bottom:.5rem;font-size:1.5rem;line-height:inherit;color:inherit;white-space:normal}progress{vertical-align:baseline}[type=number]::-webkit-inner-spin-button,[type=number]::-webkit-outer-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:none}[type=search]::-webkit-search-cancel-button,[type=search]::-webkit-search-decoration{-webkit-appearance:none}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}summary{display:list-item;cursor:pointer}template{display:none}[hidden]{display:none!important} 【略】 <style> .week-table-container{text-align:center}table.forecast-table td,table.forecast-table tr:not(.contents-title) th{text-align:center}table.forecast-table td.forecast-sentence{text-align:left;padding-left:5px;padding-right:5px}table.forecast-table p{margin-top:10px;margin-bottom:10px}table.forecast-table p.forecast-header{font-weight:700}tr.contents-header th.forecast-date{font-weight:700}div.contents-block th.forecast-area{font-weight:700}div.forecast-area{font-weight:700}th.forecast-saturday{color:#00f}th.forecast-sunday{color:red}span.forecast-max-temp,td.forecast-max-temp{color:red}span.forecast-min-temp,td.forecast-min-temp{color:#00f}td.forecast-rel-b{background-color:#dbdbdb}td.forecast-rel-c{background-color:silver}img.forecast-icon{width:60px;height:40px}table.forecast-map-panel{width:150px;background-color:#fff;border-radius:8px;box-shadow:5px 5px 5px rgba(0,0,0,.2)}table.forecast-map-panel td,table.forecast-map-panel th{text-align:center;font-size:16px;font-weight:400}th.forecast-map-panel-title{background-color:#6060c0;color:#fff;border-radius:8px 8px 0 0;white-space:normal}td.forecast-map-panel-point{white-space:normal}img.forecast-map-panel-image{width:90px;height:60px}td.forecast-map-panel-pop{padding-top:5px;border-top:solid 1px #e0e0e0}table.forecast-map-panel-en td,table.forecast-map-panel-en th{font-size:14px;line-height:1.4em}div.forecast-map-weather-100{background-color:#fa0}div.forecast-map-weather-200{background-color:#b0b0b0}div.forecast-map-weather-300{background-color:#8080ff}div.forecast-map-weather-400{background-color:#fff}div.forecast-map-weather-missing{background-color:#606060}div.forecast-map-radio-caption{display:inline-block;margin-right:9px} </style> </head> <body> <div id="header"><div class="area-pagetop" title="ページのトップへ"></div><nav class="area-nav"><div class="area-logo"><a href="/" title="トップページへ"><img src="../common/image/logo_small.svg" class="area-icon"><img src="../common/image/logo_white.svg"></a></div><div class="area-home"><a href="../" title="トップページへ"><img src="../common/image/side_home.svg" class="area-icon"></a></div><div class="area-map"><a class="clickable"><img src="../common/image/side_map.svg" title="地図へ" class="area-icon"></a></div><div class="area-dropdown"><div class="area-contents"><a id="contents-name" class="area-item-navy" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false"><img class="area-contents-menu-icon" src="../common/image/bosai_forecast.svg"><span>天気予報</span></a><div class="area-megamenu" aria-labelledby="menu_contents"><ul class="area-contents-menu"><li class="area-item-white"><a class="area-contents-menu-group-name"><span>防災</span></a></li><li class="area-item-white"><a data-context="bosai" data-url="" data-area-handover=""><span class="area-contents-menu-name">あなたの街の防災情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_home.svg" title="あなたの街の防災情報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=warning" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">気象警報・注意報</span><img class="area-contents-menu-icon" src="../common/image/bosai_warning.svg" title="気象警報・注意報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=warning_level" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">大雨危険度</span><img class="area-contents-menu-icon" src="../common/image/bosai_warning_level.svg" title="大雨危険度"></a></li><li class="area-item-white"><a data-context="bosai" data-url="risk/" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">危険度分布</span><img class="area-contents-menu-icon" src="../common/image/bosai_weathermesh.svg" title="危険度分布"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=typhoon&elem=typhoon_all" data-area-handover=""><span class="area-contents-menu-name">台風情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_typhoon.svg" title="台風情報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=information" data-area-handover="#japan=japan&offices=offices&class20s=offices"><span class="area-contents-menu-name">気象情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_information.svg" title="気象情報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#element=tornado&contents=information" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name"> 竜巻注意情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_tornado.svg" title="竜巻注意情報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#element=kirokuame&contents=information" data-area-handover="#japan=japan&offices=offices&class20s=offices"><span class="area-contents-menu-name">記録的短時間大雨情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_rain02.svg" title="記録的短時間大雨情報"></a></li><li class="area-item-white"><a data-context="jdds" data-url="//www.data.jma.go.jp/fcd/yoho/data/kouon/" data-area-handover=""><span class="area-contents-menu-name">高温注意情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_high_temp.svg" title="高温注意情報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="flood/" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">指定河川洪水予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_flood.svg" title="指定河川洪水予報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="nowc/" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">雨雲の動き</span><img class="area-contents-menu-icon" src="../common/image/bosai_rain.svg" title="雨雲の動き"></a></li><li class="area-item-white"><a data-context="bosai" data-url="kaikotan/" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">今後の雨</span><img class="area-contents-menu-icon" src="../common/image/bosai_rain.svg" title="今後の雨"></a></li><li class="area-item-white"><a data-context="bosai" data-url="snow/" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">現在の雪</span><img class="area-contents-menu-icon" src="../common/image/bosai_yukimesh.svg" title="現在の雪"></a></li></ul><ul class="area-contents-menu area-group-active"><li class="area-item-gray"><a class="area-contents-menu-group-name"><span>天気</span></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#contents=forecast" data-area-handover="#japan=japan&offices=offices&class20s=class20s" class="area-selected"><span class="area-contents-menu-name">天気予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_forecast.svg" title="天気予報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="wdist/" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">天気分布予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_wdist.svg" title="天気分布予報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="wdist/timeseries.html" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">地域時系列予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_soukei.svg" title="地域時系列予報"></a></li><li class="area-item-gray"><a data-context="jdds" data-url="//www.data.jma.go.jp/gmd/cpd/twoweek/" data-area-handover=""><span class="area-contents-menu-name">2週間気温予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_2week_forecast.svg" title="2週間気温予報"></a></li><li class="area-item-gray"><a data-context="jdds" data-url="//www.data.jma.go.jp/gmd/cpd/souten/" data-area-handover=""><span class="area-contents-menu-name">早期天候情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_soukei.svg" title="早期天候情報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#contents=season" data-area-handover="#japan=japan&offices=offices&class20s=offices"><span class="area-contents-menu-name">季節予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_season.svg" title="季節予報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="weather_map/" data-area-handover=""><span class="area-contents-menu-name">天気図</span><img class="area-contents-menu-icon" src="../common/image/bosai_weather_map.svg" title="天気図"></a></li><li class="area-item-gray"><a data-context="jdds" data-url="//www.data.jma.go.jp/gmd/env/uvindex/" data-area-handover=""><span class="area-contents-menu-name">紫外線</span><img class="area-contents-menu-icon" src="../common/image/bosai_uvidx.svg" title="紫外線"></a></li><li class="area-item-gray"><a data-context="jdds" data-url="//www.data.jma.go.jp/gmd/env/kosa/fcst/" data-area-handover=""><span class="area-contents-menu-name">黄砂</span><img class="area-contents-menu-icon" src="../common/image/bosai_kosa.svg" title="黄砂"></a></li></ul><ul class="area-contents-menu"><li class="area-item-white"><a class="area-contents-menu-group-name"><span>気象観測</span></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#elem=ir&contents=himawari" data-area-handover=""><span class="area-contents-menu-name">ひまわり</span><img class="area-contents-menu-icon" src="../common/image/bosai_himawari.svg" title="ひまわり"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#elem=temp&contents=amedas" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">アメダス</span><img class="area-contents-menu-icon" src="../common/image/bosai_amedas1h.svg" title="アメダス"></a></li><li class="area-item-white"><a data-context="jdds" data-url="//www.data.jma.go.jp/obd/bunpu/" data-area-handover=""><span class="area-contents-menu-name">推計気象分布</span><img class="area-contents-menu-icon" src="../common/image/bosai_wdist.svg" title="推計気象分布"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=windprofiler" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">ウィンドプロファイラ</span><img class="area-contents-menu-icon" src="../common/image/bosai_windprofiler.svg" title="ウィンドプロファイラ"></a></li></ul><ul class="area-contents-menu"><li class="area-item-gray"><a class="area-contents-menu-group-name"><span>海洋</span></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#contents=seawarning" data-area-handover=""><span class="area-contents-menu-name">海上警報・予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_seawarning.svg" title="海上警報・予報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="umimesh/" data-area-handover=""><span class="area-contents-menu-name">海上分布予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_umimesh.svg" title="海上分布予報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#contents=tidelevel" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">潮位観測情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_tide_height.svg" title="潮位観測 情報"></a></li><li class="area-item-gray"><a data-context="jdds" data-url="//www.data.jma.go.jp/gmd/waveinf/tile/jp/" data-area-handover=""><span class="area-contents-menu-name">波浪実況・予想図</span><img class="area-contents-menu-icon" src="../common/image/bosai_wave.svg" title="波浪実況・予想図"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#&contents=wave" data-area-handover=""><span class="area-contents-menu-name">波浪観測情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_wave.svg" title="波浪観測情報"></a></li></ul><ul class="area-contents-menu"><li class="area-item-white"><a class="area-contents-menu-group-name"><span>地震・津波</span></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=tsunami&elem=warn" data-area-handover=""><span class="area-contents-menu-name">津波</span><img class="area-contents-menu-icon" src="../common/image/bosai_tsunami.svg" title="津波"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=earthquake_map" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">地震情報</span><img class="area-contents-menu-icon" src="../common/image/bosai_earthquake.svg" title="地震情報"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=estimated_intensity_map" data-area-handover="#japan=japan&offices=offices&class20s=class20s"><span class="area-contents-menu-name">推計震度分布</span><img class="area-contents-menu-icon" src="../common/image/bosai_estimated_intensity_map.svg" title="推計震度分布"></a></li><li class="area-item-white"><a data-context="jdds" data-url="//www.data.jma.go.jp/svd/eew/data/ltpgm/" data-area-handover=""><span class="area-contents-menu-name">長周期地震動</span><img class="area-contents-menu-icon" src="../common/image/bosai_tyoushuuki.svg" title="長周期地震動"></a></li><li class="area-item-white"><a data-context="bosai" data-url="nteq" data-area-handover=""><span class="area-contents-menu-name">南海トラフ地震</span><img class="area-contents-menu-icon" src="../common/image/bosai_nteq.svg" title="南海トラフ地震"></a></li><li class="area-item-white"><a data-context="bosai" data-url="map.html#contents=hypo" data-area-handover=""><span class="area-contents-menu-name">震央分布</span><img class="area-contents-menu-icon" src="../common/image/bosai_hypo.svg" title="震央 分布"></a></li></ul><ul class="area-contents-menu"><li class="area-item-gray"><a class="area-contents-menu-group-name"><span>火山</span></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#contents=volcano" data-area-handover="#japan=japan&offices=offices&class20s=offices"><span class="area-contents-menu-name">噴火警報・噴火速報</span><img class="area-contents-menu-icon" src="../common/image/bosai_volcano.svg" title="噴火警報・噴火速 報"></a></li><li class="area-item-gray"><a data-context="bosai" data-url="map.html#contents=ashfall" data-area-handover="#japan=japan&offices=offices&class20s=offices"><span class="area-contents-menu-name">降灰予報</span><img class="area-contents-menu-icon" src="../common/image/bosai_ashfall.svg" title="降灰予報"></a></li></ul></div></div><div class="area-select" id="area-nav-cont"><ul class="area-select-ul"><li class="area-select-list"><a id="menu_office" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false" class="area-item-navy">大阪府</a><div class="area-megamenu" aria-labelledby="menu_office"><ul class="area-select-menu"><li class="area-item-white"><a>北海道</a></li><li class="area-item-white"><a id="011000">宗谷地方</a></li><li class="area-item-white"><a id="012000">上川・留萌地方</a></li><li class="area-item-white"><a id="013000">網走・北見・紋別地方</a></li><li class="area-item-white"><a id="014030">十 勝地方</a></li><li class="area-item-white"><a id="014100">釧路・根室地方</a></li><li class="area-item-white"><a id="015000">胆振・日高地方</a></li><li class="area-item-white"><a id="016000">石狩・空知・後志地方</a></li><li class="area-item-white"><a id="017000">渡島・檜山地方</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>東北</a></li><li class="area-item-gray"><a id="020000">青森県</a></li><li class="area-item-gray"><a id="030000">岩手県</a></li><li class="area-item-gray"><a id="040000">宮城県</a></li><li class="area-item-gray"><a id="050000">秋田県</a></li><li class="area-item-gray"><a id="060000">山形県</a></li><li class="area-item-gray"><a id="070000">福島県</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>関東甲信</a></li><li class="area-item-white"><a id="080000">茨城県</a></li><li class="area-item-white"><a id="090000">栃木県</a></li><li class="area-item-white"><a id="100000">群馬県</a></li><li class="area-item-white"><a id="110000">埼玉県</a></li><li class="area-item-white"><a id="120000"> 千葉県</a></li><li class="area-item-white"><a id="130000">東京都</a></li><li class="area-item-white"><a id="140000">神奈川県</a></li><li class="area-item-white"><a id="190000">山梨県</a></li><li class="area-item-white"><a id="200000">長野県</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>北陸</a></li><li class="area-item-gray"><a id="150000">新潟県</a></li><li class="area-item-gray"><a id="160000">富山県</a></li><li class="area-item-gray"><a id="170000">石川県</a></li><li class="area-item-gray"><a id="180000">福井県</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>東海</a></li><li class="area-item-white"><a id="210000">岐阜県</a></li><li class="area-item-white"><a id="220000">静岡県</a></li><li class="area-item-white"><a id="230000">愛知県</a></li><li class="area-item-white"><a id="240000">三重県</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>近畿</a></li><li class="area-item-gray"><a id="250000">滋賀県</a></li><li class="area-item-gray"><a id="260000">京都府</a></li><li class="area-item-gray"><a id="270000" class="selected">大阪府</a></li><li class="area-item-gray"><a id="280000">兵庫県</a></li><li class="area-item-gray"><a id="290000">奈良県</a></li><li class="area-item-gray"><a id="300000">和歌山県</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>中国(山口は除く)</a></li><li class="area-item-white"><a id="310000">鳥取県</a></li><li class="area-item-white"><a id="320000">島根県</a></li><li class="area-item-white"><a id="330000">岡山県</a></li><li class="area-item-white"><a id="340000">広島県</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>四国</a></li><li class="area-item-gray"><a id="360000">徳島県</a></li><li class="area-item-gray"><a id="370000">香川県</a></li><li class="area-item-gray"><a id="380000">愛媛県</a></li><li class="area-item-gray"><a id="390000">高知県</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>九州北部(山口を含む)</a></li><li class="area-item-white"><a id="350000">山口県</a></li><li class="area-item-white"><a id="400000">福岡県</a></li><li class="area-item-white"><a id="410000">佐賀県</a></li><li class="area-item-white"><a id="420000">長崎県</a></li><li class="area-item-white"><a id="430000">熊本県</a></li><li class="area-item-white"><a id="440000">大分県</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>九州南部・奄美</a></li><li class="area-item-gray"><a id="450000">宮崎県</a></li><li class="area-item-gray"><a id="460100">鹿児島県(奄美地方除く)</a></li><li class="area-item-gray"><a id="460040">奄美地方</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>沖縄</a></li><li class="area-item-white"><a id="471000">沖縄本島地方</a></li><li class="area-item-white"><a id="472000">大東島地方</a></li><li class="area-item-white"><a id="473000">宮古島地方</a></li><li class="area-item-white"><a id="474000">八重山地方</a></li></ul></div></li><li class="area-select-list"><a id="menu_class20" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false" class="area-item-navy">市町村選択</a><div class="area-megamenu" aria-labelledby="menu_class20"><ul class="area-select-menu"><li class="area-item-white"><a>[あ行] </a></li><li class="area-item-white"><a id="2720400">池田市</a></li><li class="area-item-white"><a id="2720600">泉大津市</a></li><li class="area-item-white"><a id="2721300">泉佐野市</a></li><li class="area-item-white"><a id="2721900">和泉市</a></li><li class="area-item-white"><a id="2721100">茨木市</a></li><li class="area-item-white"><a id="2723100">大阪狭山市</a></li><li class="area-item-white"><a id="2710000">大阪市</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>[か行] </a></li><li class="area-item-gray"><a id="2720800">貝塚市</a></li><li class="area-item-gray"><a id="2722100">柏原市</a></li><li class="area-item-gray"><a id="2723000">交野市</a></li><li class="area-item-gray"><a id="2722300">門真市</a></li><li class="area-item-gray"><a id="2738200">河南町</a></li><li class="area-item-gray"><a id="2721600">河内長野市</a></li><li class="area-item-gray"><a id="2720200">岸和田市</a></li><li class="area-item-gray"><a id="2736100">熊取町</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>[さ行] </a></li><li class="area-item-white"><a id="2714000">堺市</a></li><li class="area-item-white"><a id="2722900">四條畷市</a></li><li class="area-item-white"><a id="2730100">島本町</a></li><li class="area-item-white"><a id="2720500">吹田市</a></li><li class="area-item-white"><a id="2722400">摂津市</a></li><li class="area-item-white"><a id="2722800">泉南市</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>[た行] </a></li><li class="area-item-gray"><a id="2738100">太子町</a></li><li class="area-item-gray"><a id="2722500">高石市</a></li><li class="area-item-gray"><a id="2720700">高槻市</a></li><li class="area-item-gray"><a id="2736200">田尻町</a></li><li class="area-item-gray"><a id="2734100">忠岡町</a></li><li class="area-item-gray"><a id="2721800">大東市</a></li><li class="area-item-gray"><a id="2738300">千早赤阪村</a></li><li class="area-item-gray"><a id="2720300">豊中市</a></li><li class="area-item-gray"><a id="2732100">豊能町</a></li><li class="area-item-gray"><a id="2721400">富田林市</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>[な行] </a></li><li class="area-item-white"><a id="2721500">寝屋川市</a></li><li class="area-item-white"><a id="2732200">能勢町</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>[は行] </a></li><li class="area-item-gray"><a id="2722200">羽曳野市</a></li><li class="area-item-gray"><a id="2723200">阪南市</a></li><li class="area-item-gray"><a id="2722700">東大阪市</a></li><li class="area-item-gray"><a id="2721000">枚方市</a></li><li class="area-item-gray"><a id="2722600">藤井寺市</a></li></ul><ul class="area-select-menu"><li class="area-item-white"><a>[ま行] </a></li><li class="area-item-white"><a id="2721700">松原市</a></li><li class="area-item-white"><a id="2736600">岬町</a></li><li class="area-item-white"><a id="2722000">箕面市</a></li><li class="area-item-white"><a id="2720900">守口市</a></li></ul><ul class="area-select-menu"><li class="area-item-gray"><a>[や行] </a></li><li class="area-item-gray"><a id="2721200"> 八尾市</a></li></ul></div></li></ul></div></div><div class="area-gps"><a title="現在地へ"><img class="area-icon" src="../common/image/side_gps.svg"></a></div><div class="area-reload"><a class="area-update"><img src="../common/image/side_update.svg" title="更新" class="area-icon"></a></div><div class="area-sns"><a href="https://twitter.com/JMA_bousai" target="_blank" rel="noopener"><img src="../common/image/twitter_rounded.svg" title="twitter"></a></div><div class="area-search"><form id="cse-search-box" action="//google.com/cse" style="display: none;"><input type="hidden" name="q" placeholder="" style="height: 25px; width: 263px; background: url("https://www.google.com/cse/static/images/1x/ja/branding.png") left 9px top 50% no-repeat rgb(255, 255, 255);"><input type="hidden" name="cx" value="009950868052057056798:0yrbmp7chwi"><input type="hidden" name="ie" value="UTF-8"><input name="siteurl" value="www.jma.go.jp/bosai/forecast/#area_type=offices&area_code=270000" type="hidden"><input name="ref" value="" type="hidden"><input name="ss" value="" type="hidden"></form><form><div class="area-search-dropdown"></div><input type="text" placeholder="キーワー ド検索" autocomplete="off"></form><a title="検索"><img src="../common/image/side_search.svg"></a></div><div class="area-toggle"><a title="メニュー"></a></div></nav></div> <div id="forecast-area-buttons" class="container"><ul class="contents-area-breadcrumb"><li><a>全国</a></li><li>大 阪府の天気予報</li></ul></div> <div id="short-table-container" class="container"><div class="contents-block"><div class="contents-wide-table"><table class="forecast-table" style="border: none;"><tr class="contents-title"><th colspan="15">大阪府の天気予報(明後日 までの詳細)</th></tr><tr class="contents-header"><th colspan="15">2021年02月28日11時 大阪管区気象台 発表</th></tr></table><div class="contents-wide-table-body"><div class="contents-wide-table-fix" style="width: 141px; display: none;"><table class="forecast-table" style="border: none;"><tr class="simple-table-hidden-tr"><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 8.82353%; min-width: 60px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td></tr><tr class="contents-header"><th colspan="3">日付</th><th class="forecast-date forecast-sunday" colspan="4">今日 28日(日)</th><th class="forecast-date" colspan="4">明日 01日(月)</th><th class="forecast-date" colspan="4">明後日 02日(火)</th></tr><tr><th rowspan="9" class="forecast-area">大阪府</th><th rowspan="2" colspan="2">天気</th><td colspan="4"><img title="曇時々晴" src="img/201.svg"></td><td colspan="4"><img title="曇時々晴" src="img/201.svg"></td><td colspan="4"><img title="曇時々雨" src="img/202.svg"></td></tr><tr><td colspan="4" class="forecast-sentence">くもり 時々 晴れ</td><td colspan="4" class="forecast-sentence">くもり 昼前 まで 時々 晴れ</td><td colspan="4" class="forecast-sentence"> くもり 時々 雨</td></tr><tr><th colspan="2">風</th><td colspan="4" class="forecast-sentence">北東の風</td><td colspan="4" class="forecast-sentence">北東の風 海上 では 後 南の風 やや強く</td><td colspan="4" class="forecast-sentence">西の風 やや強く 後 北西の風 やや強く</td></tr><tr><th colspan="2">波</th><td colspan="4" class="forecast-sentence">0.5メートル</td><td colspan="4" class="forecast-sentence">0.5メートル 後 1メートル</td><td colspan="4" class="forecast-sentence">1.5メートル</td></tr><tr><th rowspan="2" colspan="2">降水確率(%)</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><td rowspan="2" colspan="4"></td></tr><tr><td>-</td><td>-</td><td>10</td><td>10</td><td>10</td><td>10</td><td>10</td><td>20</td></tr><tr><th rowspan="3">気温<br>(℃)</th><th></th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><td rowspan="3" colspan="4"></td></tr><tr></tr><tr><th class="forecast-area">大阪</th><td colspan="2" class="forecast-min-temp">-</td><td colspan="2" class="forecast-max-temp">15</td><td colspan="2" class="forecast-min-temp">8</td><td colspan="2" class="forecast-max-temp">19</td></tr></table></div><div class="contents-wide-table-scroll" style="margin-left: 0px; cursor: auto;"><table class="forecast-table" style="border: none; margin-left: 0px;"><tr class="simple-table-hidden-tr"><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 8.82353%; min-width: 60px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td></tr><tr class="contents-header"><th colspan="3">日付</th><th class="forecast-date forecast-sunday" colspan="4">今日 28日(日)</th><th class="forecast-date" colspan="4">明日 01日(月)</th><th class="forecast-date" colspan="4">明後日 02日(火)</th></tr><tr><th rowspan="9" class="forecast-area">大阪府</th><th rowspan="2" colspan="2">天気</th><td colspan="4"><img title="曇時々晴" src="img/201.svg"></td><td colspan="4"><img title="曇時々晴" src="img/201.svg"></td><td colspan="4"><img title="曇時々雨" src="img/202.svg"></td></tr><tr><td colspan="4" class="forecast-sentence">くもり 時々 晴れ</td><td colspan="4" class="forecast-sentence">くもり 昼前 まで 時々 晴れ</td><td colspan="4" class="forecast-sentence">くもり 時々 雨</td></tr><tr><th colspan="2">風</th><td colspan="4" class="forecast-sentence">北東の風</td><td colspan="4" class="forecast-sentence">北東の風 海上 では 後 南の風 やや強く</td><td colspan="4" class="forecast-sentence">西の風 やや強く 後 北西の風 やや 強く</td></tr><tr><th colspan="2">波</th><td colspan="4" class="forecast-sentence">0.5メートル</td><td colspan="4" class="forecast-sentence">0.5メートル 後 1メートル</td><td colspan="4" class="forecast-sentence">1.5メートル</td></tr><tr><th rowspan="2" colspan="2">降水確率(%)</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><td rowspan="2" colspan="4"></td></tr><tr><td>-</td><td>-</td><td>10</td><td>10</td><td>10</td><td>10</td><td>10</td><td>20</td></tr><tr><th rowspan="3">気温<br>(℃)</th><th></th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><td rowspan="3" colspan="4"></td></tr><tr></tr><tr><th class="forecast-area">大阪</th><td colspan="2" class="forecast-min-temp">-</td><td colspan="2" class="forecast-max-temp">15</td><td colspan="2" class="forecast-min-temp">8</td><td colspan="2" class="forecast-max-temp">19</td></tr></table></div><img src="../common/image/play_back.svg" class="contents-wide-table-icon" style="top: 134px; opacity: 0; cursor: grab;"><img src="../common/image/play_forward.svg" class="contents-wide-table-icon" style="top: 134px; cursor: grab; opacity: 0;"></div></div></div></div> <div id="short-overview-container" class="container"><div class="contents-close-box-button">▼ 大阪府の天気概況を 表示する</div><div style="display: none;"><div class="contents-block"><table class="forecast-table"><tr class="contents-title"><th>大阪府の天気概況</th></tr><tr class="contents-header"><th>2021年02月28日10時34分 大阪管区気象台 発表</th></tr><tr><td class="forecast-sentence"><p class="forecast-header">大阪府では、28日まで空気の乾燥した状態が続くため、火の取り扱いに注意してください。</p><p> 大阪府は、気圧の谷や湿った空気の影響でおおむね曇っています。<br><br> 28 日の大阪府は、気圧の谷や湿った空気の影響で、おおむね曇るでしょう。<br><br> 3月1日の大阪府は、湿った空気の影響でおおむね曇る見込みです。<br><br>【近畿地方】<br> 近畿地方は、北部では高気圧に覆われておおむね晴れていますが、中部や南部では気圧の谷や湿った空気の影響で曇っています。<br><br> 28日の近畿地方は、気圧の谷の影響でおおむね曇るでしょう。南部では湿った空気の影響で、夜は雨の降る所がある見込みです。<br><br> 3月1日の近畿地方は、湿った空気の影響でおおむね曇るでしょう。夕方からは雨の降る所がある見込みです。</p></td></tr></table></div></div></div> <div id="short-explain-container" class="container"><div class="contents-close-box-button">▼ 天気予報(明後日までの詳細)の説明を表示する</div><div class="contents-explain" style="display: none; margin-top: 10px; margin-bottom: 10px;"><div class="forecast-explain-map forecast-explain-short"> <p>毎日5時・11時・17時に発表します。</p> <p>また、この時間以外にも状況により随時発表します。</p> <p>最新の天気予報をご利用ください。</p> 【略】 <script> (function(root){"use strict";var util=JMAWebUtility;var hashC=util.hashController;var TR=Forecast.translator;var makeFweekURL=function(areas,areaType,areaCode,basePath){var code,pathCode;if(areaType==="japan"){code=areaCode}else{code=util.area.convertCode(areas,areaCode,areaType,"offices")}pathCode=Forecast.Common.getPathCode(code);return basePath+"data/forecast/"+pathCode+".json"};var insertContents=function(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,explain,areaType,areaCode,basePath){var weekAreaButtons=$("#forecast-area-buttons").empty();var shortTableContainer=$("#short-table-container").empty();var shortOverviewContainer=$("#short-overview-container").empty();var shortExplainContainer=$("#short-explain-container").empty();var weekTableContainer=$("#week-table-container").empty();var weekOverviewContainer=$("#week-overview-container").empty();var weekExplainContainer=$("#week-explain-container").empty();var fweekURL=makeFweekURL(areas,areaType,areaCode,basePath);var isDayTime;util.parallelRequest({fweek:fweekURL},function(data){var fweek=data.fweek;var hour=util.date.getHours(new Date(fweek[0].reportDatetime));ContentsCommon.makeBreadCrumbs(areaType,areaCode,TR.tr("天気予報"),areas).appendTo(weekAreaButtons);if(areaType!=="japan"){Forecast.Short.Table.insert(shortTableContainer,areas,shortAreas,anniversary,amedasTable,fweek,areaType,areaCode,basePath);if(TR.isJapanese()){Forecast.Overview.insert(shortOverviewContainer,areas,areaType,areaCode,"short")}if(hour){isDayTime=hour>=11||hour<5;ContentsCommon.makeExplain(shortExplainContainer,$(explain).filter("div.forecast-explain-short"+TR.select("","-en")),{title:TR.select("天気予報("+(isDayTime?"明後日":"明日")+"までの詳細)の説明","")})}}Forecast.Week.Table.insert(weekTableContainer,areas,weekArea,weekArea05,anniversary,amedasTable,fweek,areaType,areaCode,basePath,false);if(TR.isJapanese()){Forecast.Overview.insert(weekOverviewContainer,areas,areaType,areaCode,"week")}if(hour){isDayTime=hour>=11||hour<5;ContentsCommon.makeExplain(weekExplainContainer,areaType==="japan"?$(explain).filter("div.forecast-explain-week-japan"+TR.select("","-en")):$(explain).filter("div.forecast-explain-week"+TR.select("","-en")),{title:TR.select("天気予報 ("+(isDayTime?"7":"6")+"日先まで)の説明","")})}})};var initialize=function(){var basePath="./";util.parallelRequest({areas:"../common/const/area.json",contents:"../common/const/contents.json",weekArea:"const/week_area.json",anniversary:"const/anniversary.json",weekArea05:"const/week_area05.json",shortArea:"const/forecast_area.json",amedasTable:"../amedas/const/amedastable.json",explain:"const/explain.html"},function(data){var areas=data.areas;var contents=data.contents;var hash=fixHash(hashC.getHashData(),areas);var areaType=hash.area_type;var areaCode=hash.area_code;var weekArea=data.weekArea;var anniversary=data.anniversary;var weekArea05=data.weekArea05;var shortAreas=data.shortArea;var amedasTable=data.amedasTable;var areaMenu;areaMenu=insertAreaMenu(areas,contents,areaType,areaCode);insertContents(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,data.explain,areaType,areaCode,basePath);setEvent(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,data.explain,areaMenu,basePath)})};var fixHash=function(originalHash,areas){var hash={};var areaType=originalHash["area_type"];var areaCode=originalHash["area_code"];if((areaType==="offices"||areaType==="class20s")&&areas[areaType][areaCode]){hash["area_type"]=areaType;hash["area_code"]=areaCode}else if((areaType==="class10s"||areaType==="class15s")&&areas[areaType][areaCode]){hash["area_type"]="offices";hash["area_code"]=util.area.convertCode(areas,areaCode,areaType,"offices")}else{hash["area_type"]="japan";hash["area_code"]="010000"}return hash};var insertAreaMenu=function(areas,contents,areaType,areaCode){var areaMenuMaker=new AreaMenuMaker(areas);return areaMenuMaker.make(areaType,areaCode,TR.tr("天気予報"),"header",{map_url:"../map.html#contents=forecast",contents:contents,current_content:"forecast"})};var setEvent=function(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,explain,areaMenu,basePath){window.addEventListener("hashchange",function(){var hash=fixHash(hashC.getHashData(),areas);var areaType=hash["area_type"];var areaCode=hash["area_code"];areaMenu.setValue(hash);insertContents(areas,shortAreas,weekArea,weekArea05,anniversary,amedasTable,explain,areaType,areaCode,basePath)})};var insert=function(tag,areaType,areaCode,_officeCode,options){util.parallelRequest({weekArea:"forecast/const/week_area.json",anniversary:"forecast/const/anniversary.json",weekArea05:"forecast/const/week_area05.json",amedasTable:"amedas/const/amedastable.json"},function(data){var weekTableContainer=$(tag);var areas=options.areas;var weekArea=data.weekArea;var anniversary=data.anniversary;var weekArea05=data.weekArea05;var amedasTable=data.amedasTable;var basePath="forecast/";var shortened=true;var fweekURL=makeFweekURL(areas,areaType,areaCode,basePath);util.parallelRequest({fweek:fweekURL},function(data2){var fweek=data2.fweek;Forecast.Week.Table.insert(weekTableContainer,areas,weekArea,weekArea05,anniversary,amedasTable,fweek,areaType,areaCode,basePath,shortened)})})};root.Forecast=root.Forecast||{};root.Forecast.initialize=initialize;root.Forecast.insert=insert})(typeof window==="object"?window:typeof global==="object"?global:this); </script> <script> $(function () { Forecast.initialize(); }); </script> <script src="//www.google.com/cse/brand?form=cse-search-box&lang=ja" id="google-script" type="application/javascript"></script></body></html> |
[スポンサーリンク]
Requests-HTMLライブラリでパース
Requests-HTMLライブラリはパースライブラリです
取得したHTMLをパースしてみます
HTMLタグやCSSセレクタでのパースに対応しています
Requests-HTMLライブラリでパース サンプルコード
・CSSセレクタを指定して要素を取得
#でCSSセレクタを指定して要素を取得
1 |
ShortTableContainer = ScrapingGetHtmlResponse.html.find('#short-table-container') |
・HTMLタグを指定して要素を取得
1 |
TableHtmlTag = ScrapingGetHtmlResponse.html.find('table') |
・取得した要素のHTMLを取得
1 |
ShortTableContainer[0].html |
・取得した要素のTEXTを取得
1 |
ShortTableContainer[0].text |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
# coding: utf_8 # モジュールインポート from requests_html import HTMLSession # WEBスクレイピングパラメータ設定 ScrapingGetHtmlUrl = 'https://www.jma.go.jp/bosai/forecast/#area_type=offices&area_code=270000' # WEBスクレイピング実行 ScrapingSession = HTMLSession() ScrapingGetHtmlResponse = ScrapingSession.get(ScrapingGetHtmlUrl) ScrapingGetHtmlResponse.html.render(timeout=0,sleep=10) if ScrapingGetHtmlResponse.status_code == 200: ShortTableContainer = ScrapingGetHtmlResponse.html.find('#short-table-container') print("=====CSS id=short-table-container 取得要素=====") print(ShortTableContainer[0]) print("\n\n") print("=====CSS id=short-table-container HTML=====") print(ShortTableContainer[0].html) print("\n\n") print("=====CSS id=short-table-container TEXT=====") print(ShortTableContainer[0].text) TableHtmlTag = ScrapingGetHtmlResponse.html.find('table') print("\n\n") print("=====HTML table 取得要素=====") print(TableHtmlTag[0]) print("\n\n") print("=====HTML table HTML=====") print(TableHtmlTag[0].html) print("\n\n") print("=====HTML table TEXT=====") print(TableHtmlTag[0].text) else: print("WEBスクレイピングに失敗") print(ScrapingGetHtmlResponse) |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 |
=====CSS id=short-table-container 取得要素===== <Element 'div' id='short-table-container' class=('container',)> =====CSS id=short-table-container HTML===== <div id="short-table-container" class="container"><div class="contents-block"><div class="contents-wide-table"><table class="forecast-table" style="border: none;"><tr class="contents-title"><th colspan="15">大阪府の天気予報(明後日まで の詳細)</th></tr><tr class="contents-header"><th colspan="15">2021年02月28日17時 大阪管区気象台 発表</th></tr></table><div class="contents-wide-table-body"><div class="contents-wide-table-fix" style="width: 141px; display: none;"><table class="forecast-table" style="border: none;"><tr class="simple-table-hidden-tr"><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"/><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"/><td style="width: 8.82353%; min-width: 60px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/></tr><tr class="contents-header"><th colspan="3">日付</th><th class="forecast-date forecast-sunday" colspan="4">今夜 28日(日)</th><th class="forecast-date" colspan="4">明日 01日(月)</th><th class="forecast-date" colspan="4">明後日 02日(火)</th></tr><tr><th rowspan="9" class="forecast-area">大阪府</th><th rowspan="2" colspan="2">天気</th><td colspan="4"><img title="曇時々晴" src="img/601.svg"/></td><td colspan="4"><img title="曇時々晴" src="img/201.svg"/></td><td colspan="4"><img title="曇時々雨" src="img/202.svg"/></td></tr><tr><td colspan="4" class="forecast-sentence">くもり 夜のはじめ頃 晴れ</td><td colspan="4" class="forecast-sentence">くもり 昼前 まで 時々 晴れ</td><td colspan="4" class="forecast-sentence">くもり 時々 雨</td></tr><tr><th colspan="2">風</th><td colspan="4" class="forecast-sentence">北東 の風</td><td colspan="4" class="forecast-sentence">北東の風 海上 では 後 南の風 やや強く</td><td colspan="4" class="forecast-sentence">南の風 やや強く 後 北西の風 やや強く</td></tr><tr><th colspan="2">波</th><td colspan="4" class="forecast-sentence">0.5メートル</td><td colspan="4" class="forecast-sentence">0.5メートル 後 1メートル</td><td colspan="4" class="forecast-sentence">1.5メートル 後 2メートル</td></tr><tr><th rowspan="2" colspan="2">降 水確率(%)</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><td rowspan="2" colspan="4"/></tr><tr><td>-</td><td>-</td><td>-</td><td>10</td><td>10</td><td>10</td><td>10</td><td>20</td></tr><tr><th rowspan="3">気温<br/>(℃)</th><th/><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><td rowspan="3" colspan="4"/></tr><tr/><tr><th class="forecast-area">大阪</th><td colspan="2" class="forecast-min-temp">-</td><td colspan="2" class="forecast-max-temp">-</td><td colspan="2" class="forecast-min-temp">8</td><td colspan="2" class="forecast-max-temp">20</td></tr></table></div><div class="contents-wide-table-scroll" style="margin-left: 0px; cursor: auto;"><table class="forecast-table" style="border: none; margin-left: 0px;"><tr class="simple-table-hidden-tr"><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"/><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"/><td style="width: 8.82353%; min-width: 60px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"/></tr><tr class="contents-header"><th colspan="3">日付</th><th class="forecast-date forecast-sunday" colspan="4">今夜 28日(日)</th><th class="forecast-date" colspan="4">明日 01日(月)</th><th class="forecast-date" colspan="4">明後日 02日(火)</th></tr><tr><th rowspan="9" class="forecast-area">大阪府</th><th rowspan="2" colspan="2">天気</th><td colspan="4"><img title="曇時々晴" src="img/601.svg"/></td><td colspan="4"><img title="曇時々晴" src="img/201.svg"/></td><td colspan="4"><img title="曇時々雨" src="img/202.svg"/></td></tr><tr><td colspan="4" class="forecast-sentence">くもり 夜のはじめ頃 晴れ</td><td colspan="4" class="forecast-sentence">くもり 昼前 まで 時々 晴れ</td><td colspan="4" class="forecast-sentence">くもり 時々 雨</td></tr><tr><th colspan="2">風</th><td colspan="4" class="forecast-sentence">北東の風</td><td colspan="4" class="forecast-sentence">北東の風 海上 では 後 南の風 やや強く</td><td colspan="4" class="forecast-sentence">南の風 やや強く 後 北西の風 やや強く</td></tr><tr><th colspan="2">波</th><td colspan="4" class="forecast-sentence">0.5メートル</td><td colspan="4" class="forecast-sentence">0.5メートル 後 1メートル</td><td colspan="4" class="forecast-sentence">1.5メートル 後 2メートル</td></tr><tr><th rowspan="2" colspan="2">降水確率(%)</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><td rowspan="2" colspan="4"/></tr><tr><td>-</td><td>-</td><td>-</td><td>10</td><td>10</td><td>10</td><td>10</td><td>20</td></tr><tr><th rowspan="3">気温<br/>(℃)</th><th/><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><td rowspan="3" colspan="4"/></tr><tr/><tr><th class="forecast-area">大阪</th><td colspan="2" class="forecast-min-temp">-</td><td colspan="2" class="forecast-max-temp">-</td><td colspan="2" class="forecast-min-temp">8</td><td colspan="2" class="forecast-max-temp">20</td></tr></table></div><img src="../common/image/play_back.svg" class="contents-wide-table-icon" style="top: 134px; opacity: 0; cursor: grab;"/><img src="../common/image/play_forward.svg" class="contents-wide-table-icon" style="top: 134px; cursor: grab; opacity: 0;"/></div></div></div></div> =====CSS id=short-table-container TEXT===== 大阪府の天気予報(明後日までの詳細) 2021年02月28日17時 大阪管区気象台 発表 日付 今夜 28日(日) 明日 01日(月) 明後日 02日(火) 大阪府 天気 くもり 夜のはじめ頃 晴れ くもり 昼前 まで 時々 晴れ くもり 時々 雨 風 北東の風 北東の風 海上 では 後 南の風 やや強く 南の風 やや強く 後 北西の風 やや強く 波 0.5メートル 0.5メートル 後 1メートル 1.5メートル 後 2メートル 降水確率(%) 00-06 06-12 12-18 18-24 00-06 06-12 12-18 18-24 - - - 10 10 10 10 20 気温 (℃) 朝の最低 日中の最高 朝の最低 日中の最高 大阪 - - 8 20 日付 今夜 28日(日) 明日 01日(月) 明後日 02日(火) 大阪府 天気 くもり 夜のはじめ頃 晴れ くもり 昼前 まで 時々 晴れ くもり 時々 雨 風 北東の風 北東の風 海上 では 後 南の風 やや強く 南の風 やや強く 後 北西の風 やや強く 波 0.5メートル 0.5メートル 後 1メートル 1.5メートル 後 2メートル 降水確率(%) 00-06 06-12 12-18 18-24 00-06 06-12 12-18 18-24 - - - 10 10 10 10 20 気温 (℃) 朝の最低 日中の最高 朝の最低 日中の最高 大阪 - - 8 20 =====HTML table 取得要素===== <Element 'table' class=('forecast-table',) style='border: none;'> =====HTML table HTML===== <table class="forecast-table" style="border: none;"><tr class="contents-title"><th colspan="15">大阪府の天気予報(明後日までの詳細)</th></tr><tr class="contents-header"><th colspan="15">2021年02月28日17時 大阪管区気象台 発表</th></tr></table> =====HTML table TEXT===== 大阪府の天気予報(明後日までの詳細) 2021年02月28日17時 大阪管区気象台 発表 |
Requests-HTMLライブラリでコードを取得してBeautiful Soupライブラリでパース サンプルコード
Requests-HTMLライブラリを使ってHTMLコードを取得することが出来ます
HTMLコードが取得出来ているので、Beautiful Soupライブラリでパースすることも可能です
下段のサンプルコードと実行結果の様に、上述のRequests-HTMLライブラリのパースと同じ様な値を取得することも出来ます
・Requests-HTMLで取得した結果をBeautiful SoupでHTML形式でパース
1 |
ScrapingGetHtmlResponseHtml = BeautifulSoup(ScrapingGetHtmlResponse.html.html, 'html.parser') |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
# coding: utf_8 # モジュールインポート from requests_html import HTMLSession from bs4 import BeautifulSoup # WEBスクレイピングパラメータ設定 ScrapingGetHtmlUrl = 'https://www.jma.go.jp/bosai/forecast/#area_type=offices&area_code=270000' # WEBスクレイピング実行 ScrapingSession = HTMLSession() ScrapingGetHtmlResponse = ScrapingSession.get(ScrapingGetHtmlUrl) ScrapingGetHtmlResponse.html.render(timeout=0,sleep=10) ScrapingGetHtmlResponseHtml = BeautifulSoup(ScrapingGetHtmlResponse.html.html, 'html.parser') if ScrapingGetHtmlResponse.status_code == 200: ShortTableContainer = ScrapingGetHtmlResponseHtml.find_all('div', {'class': 'contents-wide-table-body'}) print("=====CSS id=short-table-container HTML=====") print(ShortTableContainer[0]) print("\n\n") print("=====CSS id=short-table-container TEXT=====") print(ShortTableContainer[0].text) print("\n\n") TableHtmlTag = ScrapingGetHtmlResponseHtml.find_all('table') print("=====HTML table HTML=====") print(TableHtmlTag[0]) print("\n\n") print("=====HTML table TEXT=====") print(TableHtmlTag[0].text) else: print("WEBスクレイピングに失敗") print(ScrapingGetHtmlResponse) |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
=====CSS id=short-table-container HTML===== <div class="contents-wide-table-body"><div class="contents-wide-table-fix" style="width: 141px; display: none;"><table class="forecast-table" style="border: none;"><tr class="simple-table-hidden-tr"><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 8.82353%; min-width: 60px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td></tr><tr class="contents-header"><th colspan="3">日付</th><th class="forecast-date forecast-sunday" colspan="4">今夜 28日(日)</th><th class="forecast-date" colspan="4">明日 01日(月)</th><th class="forecast-date" colspan="4">明後日 02日(火)</th></tr><tr><th class="forecast-area" rowspan="9">大阪府</th><th colspan="2" rowspan="2">天気</th><td colspan="4"><img src="img/601.svg" title="曇時々晴"/></td><td colspan="4"><img src="img/201.svg" title="曇時々晴"/></td><td colspan="4"><img src="img/202.svg" title="曇時々雨"/></td></tr><tr><td class="forecast-sentence" colspan="4">くもり 夜のはじめ頃 晴れ</td><td class="forecast-sentence" colspan="4">くもり 昼前 まで 時々 晴れ</td><td class="forecast-sentence" colspan="4">くもり 時々 雨</td></tr><tr><th colspan="2">風</th><td class="forecast-sentence" colspan="4">北東の風</td><td class="forecast-sentence" colspan="4">北東の風 海上 では 後 南の風 やや強く</td><td class="forecast-sentence" colspan="4">南の風 やや強く 後 北西の風 やや強く</td></tr><tr><th colspan="2">波</th><td class="forecast-sentence" colspan="4">0.5メートル</td><td class="forecast-sentence" colspan="4">0.5メートル 後 1メートル</td><td class="forecast-sentence" colspan="4">1.5メートル 後 2メートル</td></tr><tr><th colspan="2" rowspan="2">降水確率(%)</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><td colspan="4" rowspan="2"></td></tr><tr><td>-</td><td>-</td><td>-</td><td>10</td><td>10</td><td>10</td><td>10</td><td>20</td></tr><tr><th rowspan="3">気温<br/>(℃)</th><th></th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><td colspan="4" rowspan="3"></td></tr><tr></tr><tr><th class="forecast-area">大阪</th><td class="forecast-min-temp" colspan="2">-</td><td class="forecast-max-temp" colspan="2">-</td><td class="forecast-min-temp" colspan="2">8</td><td class="forecast-max-temp" colspan="2">20</td></tr></table></div><div class="contents-wide-table-scroll" style="margin-left: 0px; cursor: auto;"><table class="forecast-table" style="border: none; margin-left: 0px;"><tr class="simple-table-hidden-tr"><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 5.88235%; min-width: 40px; padding: 0px; border-bottom: hidden;"></td><td style="width: 8.82353%; min-width: 60px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td><td style="width: 6.61765%; min-width: 45px; padding: 0px; border-bottom: hidden;"></td></tr><tr class="contents-header"><th colspan="3">日付</th><th class="forecast-date forecast-sunday" colspan="4">今夜 28日(日)</th><th class="forecast-date" colspan="4">明日 01日(月)</th><th class="forecast-date" colspan="4">明後日 02日(火)</th></tr><tr><th class="forecast-area" rowspan="9">大阪府</th><th colspan="2" rowspan="2">天気</th><td colspan="4"><img src="img/601.svg" title="曇時々晴"/></td><td colspan="4"><img src="img/201.svg" title="曇時々晴"/></td><td colspan="4"><img src="img/202.svg" title="曇時々雨"/></td></tr><tr><td class="forecast-sentence" colspan="4">くもり 夜のはじめ頃 晴れ</td><td class="forecast-sentence" colspan="4">くもり 昼前 まで 時々 晴れ</td><td class="forecast-sentence" colspan="4">くもり 時々 雨</td></tr><tr><th colspan="2">風</th><td class="forecast-sentence" colspan="4">北東の風</td><td class="forecast-sentence" colspan="4">北東の風 海上 では 後 南の風 やや強く</td><td class="forecast-sentence" colspan="4">南の風 やや強く 後 北西の風 やや強く</td></tr><tr><th colspan="2">波</th><td class="forecast-sentence" colspan="4">0.5 メートル</td><td class="forecast-sentence" colspan="4">0.5メートル 後 1メートル</td><td class="forecast-sentence" colspan="4">1.5メートル 後 2メートル</td></tr><tr><th colspan="2" rowspan="2">降水確率(%)</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><th>00-06</th><th>06-12</th><th>12-18</th><th>18-24</th><td colspan="4" rowspan="2"></td></tr><tr><td>-</td><td>-</td><td>-</td><td>10</td><td>10</td><td>10</td><td>10</td><td>20</td></tr><tr><th rowspan="3">気温<br/>(℃)</th><th></th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><th colspan="2">朝の最低</th><th colspan="2">日中の最高</th><td colspan="4" rowspan="3"></td></tr><tr></tr><tr><th class="forecast-area">大阪</th><td class="forecast-min-temp" colspan="2">-</td><td class="forecast-max-temp" colspan="2">-</td><td class="forecast-min-temp" colspan="2">8</td><td class="forecast-max-temp" colspan="2">20</td></tr></table></div><img class="contents-wide-table-icon" src="../common/image/play_back.svg" style="top: 134px; opacity: 0; cursor: grab;"/><img class="contents-wide-table-icon" src="../common/image/play_forward.svg" style="top: 134px; cursor: grab; opacity: 0;"/></div> =====CSS id=short-table-container TEXT===== 日付今夜 28日(日)明日 01日(月)明後日 02日(火)大阪府天気くもり 夜のはじめ頃 晴れくもり 昼前 まで 時々 晴れくもり 時々 雨風北東の風北東の風 海上 では 後 南の風 やや強く南の風 やや強く 後 北西の風 やや強く波0.5メートル0.5メートル 後 1メートル1.5メートル 後 2メートル降水確率(%)00-0606-1212-1818-2400-0606-1212-1818-24---1010101020気温(℃)朝の最低日中の最高朝の最低日中の最高大阪--820日付今夜 28日(日)明日 01日(月)明後日 02日(火)大阪府天気く もり 夜のはじめ頃 晴れくもり 昼前 まで 時々 晴れくもり 時々 雨風北東の風北東の風 海上 では 後 南の風 やや強く南の風 やや強く 後 北西の風 やや強く波0.5メートル0.5メートル 後 1メートル1.5メートル 後 2メートル降水確率(%)00-0606-1212-1818-2400-0606-1212-1818-24---1010101020気温(℃)朝の最低日中の最高朝の最低日中の最高大阪--820 =====HTML table HTML===== <table class="forecast-table" style="border: none;"><tr class="contents-title"><th colspan="15">大阪府の天気予報(明後日までの詳細)</th></tr><tr class="contents-header"><th colspan="15">2021年02月28日17時 大阪管区気象台 発表</th></tr></table> =====HTML table TEXT===== 大阪府の天気予報(明後日までの詳細)2021年02月28日17時 大阪管区気象台 発表 |
[スポンサーリンク]
コメント