diff --git a/CNAME b/CNAME new file mode 100644 index 0000000..7abda01 --- /dev/null +++ b/CNAME @@ -0,0 +1 @@ +python-tutorials.siteleaf.net \ No newline at end of file diff --git a/LICENSE b/LICENSE deleted file mode 100644 index 66a14d7..0000000 --- a/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2017 JCutrer (https://jcutrer.com) - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. \ No newline at end of file diff --git a/net/automation/RouterOS-Automation-with-NAPALM.ipynb b/net/automation/RouterOS-Automation-with-NAPALM.ipynb deleted file mode 100644 index 071eb47..0000000 --- a/net/automation/RouterOS-Automation-with-NAPALM.ipynb +++ /dev/null @@ -1,778 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# MikroTik RouterOS Automation with NAPALM (python module)\n", - "\n", - "In this tutorial, we will explore using the NAPALM python module to query data from a MikroTik Router.\n", - "\n", - "Before we begin, you are expected to have python3 and pip installed as well as access to a MikroTik router running RouterOS. NAPALM will attempt to connect to the router on the default API port of 8728. You will need to enable the API service which is found in `IP | Services` using winbox\n", - "\n", - "At the time of writing this article, the RouterOS NAPALM driver does not support configuration management for MikroTik routers. For now, we will look at what information can be read from the router.\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Here are the functions that NAPALM-ROS currently supports\n", - "\n", - "> According to the [napalm-ros github page](https://github.com/napalm-automation-community/napalm-ros)\n", - "\n", - "* `get_arp_table`\n", - "* `get_interfaces_counters`\n", - "* `get_environment`\n", - "* `get_facts`\n", - "* `get_interfaces`\n", - "* `get_interfaces_ip`\n", - "* `get_ntp_servers`\n", - "* `get_snmp_information`\n", - "* `get_users`\n", - "* `get_ipv6_neighbors_table`\n", - "* `is_alive`\n", - "* `ping`" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Create a new pipenv virtualenv and install NAPALM and the RouterOS driver\n", - "\n", - "```shell\n", - "pipenv --python 3.6\n", - "pipenv install napalm\n", - "pipenv install napalm-ros\n", - "``` \n", - "\n", - "### ... or Globally Install NAPALM and the RouterOS driver using pip\n", - "\n", - "If you have not idea what pipenv is, checkout my [pipenv tutorial](https://jcutrer.com/howto/dev/python/pipenv-pipfile) or alternatively you can install `napalm` & `napalm-ros` globally with `pip`.\n", - "\n", - "```shell\n", - "pip install napalm\n", - "pip install napalm-ros\n", - "```\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Import NAPALM and the RouterOS driver" - ] - }, - { - "cell_type": "code", - "execution_count": 215, - "metadata": {}, - "outputs": [], - "source": [ - "import napalm\n", - "from napalm_ros import ros" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Provide your MikroTik Router's IP and credentials" - ] - }, - { - "cell_type": "code", - "execution_count": 216, - "metadata": {}, - "outputs": [], - "source": [ - "router_ip = '192.168.1.1'\n", - "router_port = 8728 # Use 8729 for api-ssl\n", - "router_user = 'admin'\n", - "router_pass = ''" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Configure the RouterOS driver then initialize and connect to the router" - ] - }, - { - "cell_type": "code", - "execution_count": 217, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Connecting to 192.168.1.1 on port 8728 as admin\n", - "Opening ...\n" - ] - } - ], - "source": [ - "# Use the RouterOS (ros) network driver to connect to the device:\n", - "driver = napalm.get_network_driver('ros')\n", - "\n", - "\n", - "print('Connecting to', router_ip, \"on port\", router_port, \"as\", router_user)\n", - "\n", - "device = driver(hostname=router_ip, username=router_user,\n", - " password=router_pass, optional_args={'port': router_port})\n", - "\n", - "print('Opening ...')\n", - "device.open()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### List available methods of the RouterOS NAPALM driver" - ] - }, - { - "cell_type": "code", - "execution_count": 218, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "device.api()\n", - "device.cli()\n", - "device.close()\n", - "device.commit_config()\n", - "device.compare_config()\n", - "device.compliance_report()\n", - "device.connection_tests()\n", - "device.discard_config()\n", - "device.get_arp_table()\n", - "device.get_bgp_config()\n", - "device.get_bgp_neighbors()\n", - "device.get_bgp_neighbors_detail()\n", - "device.get_config()\n", - "device.get_environment()\n", - "device.get_facts()\n", - "device.get_firewall_policies()\n", - "device.get_interfaces()\n", - "device.get_interfaces_counters()\n", - "device.get_interfaces_ip()\n", - "device.get_ipv6_neighbors_table()\n", - "device.get_lldp_neighbors()\n", - "device.get_lldp_neighbors_detail()\n", - "device.get_mac_address_table()\n", - "device.get_network_instances()\n", - "device.get_ntp_peers()\n", - "device.get_ntp_servers()\n", - "device.get_ntp_stats()\n", - "device.get_optics()\n", - "device.get_probes_config()\n", - "device.get_probes_results()\n", - "device.get_route_to()\n", - "device.get_snmp_information()\n", - "device.get_users()\n", - "device.is_alive()\n", - "device.load_merge_candidate()\n", - "device.load_replace_candidate()\n", - "device.load_template()\n", - "device.open()\n", - "device.ping()\n", - "device.post_connection_tests()\n", - "device.pre_connection_tests()\n", - "device.rollback()\n", - "device.traceroute()\n" - ] - } - ], - "source": [ - "method_list = [func for func in dir(device) if callable(getattr(device, func)) and not func.startswith(\"_\")]\n", - "\n", - "for method in method_list:\n", - " print(f\"device.{method}()\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As previously mentioned, some of these method such as `.get_config()` are not implemented (yet)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### To begin, let's see if we are connected to the router?" - ] - }, - { - "cell_type": "code", - "execution_count": 219, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'is_alive': True}\n" - ] - } - ], - "source": [ - "print(device.is_alive())" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Ping from the router" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To be clear, the `ping()` function performs an icmp **ping from the router** to some destination IP, not from your computer to the router." - ] - }, - { - "cell_type": "code", - "execution_count": 220, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'success': {'probes_sent': 5, 'packet_loss': 0, 'rtt_min': 50.0, 'rtt_max': 69.0, 'rtt_avg': 54.0, 'rtt_stddev': -1.0, 'results': [{'ip_address': '8.8.8.8', 'rtt': 69.0}, {'ip_address': '8.8.8.8', 'rtt': 51.0}, {'ip_address': '8.8.8.8', 'rtt': 50.0}, {'ip_address': '8.8.8.8', 'rtt': 50.0}, {'ip_address': '8.8.8.8', 'rtt': 51.0}]}}\n" - ] - } - ], - "source": [ - "print(device.ping('8.8.8.8'))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "A more advanced ping example with additional arguments" - ] - }, - { - "cell_type": "code", - "execution_count": 221, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'success': {'probes_sent': 3, 'packet_loss': 0, 'rtt_min': 50.0, 'rtt_max': 52.0, 'rtt_avg': 51.0, 'rtt_stddev': -1.0, 'results': [{'ip_address': '8.8.8.8', 'rtt': 52.0}, {'ip_address': '8.8.8.8', 'rtt': 51.0}, {'ip_address': '8.8.8.8', 'rtt': 50.0}]}}\n" - ] - } - ], - "source": [ - "print(device.ping(destination='8.8.8.8', source='192.168.1.1', ttl=255, timeout=1000, size=64, count=3))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Get SNMP Configuration" - ] - }, - { - "cell_type": "code", - "execution_count": 222, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'chassis_id': '', 'community': {'public': {'acl': '0.0.0.0/0', 'mode': 'ro'}}, 'contact': '', 'location': ''}\n" - ] - } - ], - "source": [ - "print(device.get_snmp_information())" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Dump the Router's Facts" - ] - }, - { - "cell_type": "code", - "execution_count": 150, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'uptime': 121823, 'vendor': 'MikroTik', 'model': 'RB951G-2HnD', 'hostname': 'R1', 'fqdn': '', 'os_version': '6.41.1 (stable)', 'serial_number': '3XXE021XXXXX', 'interface_list': ['bridge', 'ether1', 'ether2', 'ether3', 'ether4', 'ether5', 'wlan1', 'wlan2']}\n" - ] - } - ], - "source": [ - "print(device.get_facts())" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Iterate over the Router Facts and print them" - ] - }, - { - "cell_type": "code", - "execution_count": 212, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Facts about 192.168.1.1\n", - "uptime: 124775\n", - "vendor: MikroTik\n", - "model: RB951G-2HnD\n", - "hostname: R1\n", - "fqdn: \n", - "os_version: 6.41.1 (stable)\n", - "serial_number: 3XXE021XXXXX\n", - "interface_list: ['bridge', 'ether1', 'ether2', 'ether3', 'ether4', 'ether5', 'wlan1', 'wlan2']\n" - ] - } - ], - "source": [ - "print(\"Facts about\", router_ip)\n", - "\n", - "for key, value in device.get_facts().items():\n", - " print( f\"{key}: {value}\" )" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Iterate over the router facts's interface list" - ] - }, - { - "cell_type": "code", - "execution_count": 152, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "List of interfaces on this router\n", - "bridge\n", - "ether1\n", - "ether2\n", - "ether3\n", - "ether4\n", - "ether5\n", - "wlan1\n", - "wlan2\n" - ] - } - ], - "source": [ - "print(\"List of interfaces on this router\")\n", - "\n", - "for interf in device.get_facts()['interface_list']:\n", - " print( interf )" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Query the router's interface counter statistics" - ] - }, - { - "cell_type": "code", - "execution_count": 192, - "metadata": {}, - "outputs": [], - "source": [ - "iface_stats = device.get_interfaces_counters()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Dump the Interfaces Stats" - ] - }, - { - "cell_type": "code", - "execution_count": 193, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'bridge': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 1250539661,\n", - " 'rx_unicast_packets': 6170632,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 19929132599,\n", - " 'tx_unicast_packets': 14415771}),\n", - " 'ether1': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 19983333109,\n", - " 'rx_unicast_packets': 14369741,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 1252233599,\n", - " 'tx_unicast_packets': 6028405}),\n", - " 'ether2': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 104970493,\n", - " 'rx_unicast_packets': 155997,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 35736133,\n", - " 'tx_unicast_packets': 270735}),\n", - " 'ether3': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 0,\n", - " 'rx_unicast_packets': 0,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 0,\n", - " 'tx_unicast_packets': 0}),\n", - " 'ether4': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 7490684,\n", - " 'rx_unicast_packets': 17948,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 22773137,\n", - " 'tx_unicast_packets': 160526}),\n", - " 'ether5': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 635334841,\n", - " 'rx_unicast_packets': 2018650,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 5036923454,\n", - " 'tx_unicast_packets': 4069320}),\n", - " 'wlan1': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 0,\n", - " 'rx_unicast_packets': 0,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 0,\n", - " 'tx_unicast_packets': 0}),\n", - " 'wlan2': defaultdict(int,\n", - " {'rx_broadcast_packets': 0,\n", - " 'rx_discards': 0,\n", - " 'rx_errors': 0,\n", - " 'rx_multicast_packets': 0,\n", - " 'rx_octets': 535541903,\n", - " 'rx_unicast_packets': 4070323,\n", - " 'tx_broadcast_packets': 0,\n", - " 'tx_discards': 0,\n", - " 'tx_errors': 0,\n", - " 'tx_multicast_packets': 0,\n", - " 'tx_octets': 14832924157,\n", - " 'tx_unicast_packets': 10469485})}" - ] - }, - "execution_count": 193, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "iface_stats" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Dump a single interface's stats" - ] - }, - { - "cell_type": "code", - "execution_count": 224, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "defaultdict(, {'tx_errors': 0, 'rx_errors': 0, 'tx_discards': 0, 'rx_discards': 0, 'tx_octets': 1252233599, 'rx_octets': 19983333109, 'tx_unicast_packets': 6028405, 'rx_unicast_packets': 14369741, 'tx_multicast_packets': 0, 'rx_multicast_packets': 0, 'tx_broadcast_packets': 0, 'rx_broadcast_packets': 0})\n" - ] - } - ], - "source": [ - "print( iface_stats['ether1'])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Iterate over an interface's stats and print them" - ] - }, - { - "cell_type": "code", - "execution_count": 228, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "ether1 Interface Stats\n", - "======================\n", - "tx_errors: 0\n", - "rx_errors: 0\n", - "tx_discards: 0\n", - "rx_discards: 0\n", - "tx_octets: 1252233599\n", - "rx_octets: 19983333109\n", - "tx_unicast_packets: 6028405\n", - "rx_unicast_packets: 14369741\n", - "tx_multicast_packets: 0\n", - "rx_multicast_packets: 0\n", - "tx_broadcast_packets: 0\n", - "rx_broadcast_packets: 0\n" - ] - } - ], - "source": [ - "print( \"ether1 Interface Stats\")\n", - "print(\"======================\")\n", - "\n", - "for stat in iface_stats['ether1']:\n", - " print(f\"{stat}: {iface_stats['ether1'][stat]}\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### List users" - ] - }, - { - "cell_type": "code", - "execution_count": 197, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'admin': {'level': 15, 'password': '', 'sshkeys': []}}\n" - ] - } - ], - "source": [ - "users = device.get_users()\n", - "\n", - "print(users)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Get all ARP Entries" - ] - }, - { - "cell_type": "code", - "execution_count": 158, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'interface': 'bridge', 'mac': 'XX:00:0B:XX:XX:XX', 'ip': '192.168.1.3', 'age': -1.0}\n", - "{'interface': 'bridge', 'mac': 'XX:9F:C2:XX:XX:XX', 'ip': '192.168.1.4', 'age': -1.0}\n", - "{'interface': 'bridge', 'mac': 'XX:7C:9C:XX:XX:XX', 'ip': '192.168.1.5', 'age': -1.0}\n" - ] - } - ], - "source": [ - "arptable = device.get_arp_table()\n", - "\n", - "for entry in arptable:\n", - " print(entry)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Get Interface Information" - ] - }, - { - "cell_type": "code", - "execution_count": 229, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Print a list of interfaces with w/comments & status\n", - ";;; WAN\n", - "ether1 [enabled=True, up=True]\n", - "\n", - ";;; Uplink to DVR\n", - "ether2 [enabled=True, up=True]\n", - "\n", - ";;; \n", - "ether3 [enabled=False, up=False]\n", - "\n", - ";;; uplink to NAS\n", - "ether4 [enabled=True, up=True]\n", - "\n", - ";;; UniFi AP\n", - "ether5 [enabled=True, up=True]\n", - "\n", - ";;; \n", - "wlan1 [enabled=True, up=False]\n", - "\n", - ";;; \n", - "wlan2 [enabled=True, up=True]\n", - "\n", - ";;; defconf\n", - "bridge [enabled=True, up=True]\n", - "\n" - ] - } - ], - "source": [ - "ifaces = device.get_interfaces()\n", - "\n", - "print(\"Print a list of interfaces with w/comments & status\")\n", - "\n", - "for iface,data in ifaces.items():\n", - " print(f\";;; {data['description']}\")\n", - " print(f\"{iface} [enabled={data['is_enabled']}, up={data['is_up']}]\")\n", - " print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Gracefully disconnect from the Router" - ] - }, - { - "cell_type": "code", - "execution_count": 160, - "metadata": {}, - "outputs": [], - "source": [ - "device.close()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "I hope you enjoyed this **python tutorial** about **NAPALM** and **RouterOS Automation**. Checkout my other [MikroTik tutorials](https://jcutrer.com/howto/networking/mikrotik/) and [Python tutorials](https://jcutrer.com/howto/dev/python/)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This jupyter notbook was created by [JCutrer](https://jcutrer.com) and the contained code is provided under a [MIT License](). If you use or reference this code please consider linking back to the companion [python tutorial](https://jcutrer.com/howto/networking/mikrotik/napalm-routeros-automation) " - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.4" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/web/rss/How to Parse and Combine RSS News headlines using feedparser.ipynb b/web/rss/How to Parse and Combine RSS News headlines using feedparser.ipynb index c48838a..eb40f3d 100644 --- a/web/rss/How to Parse and Combine RSS News headlines using feedparser.ipynb +++ b/web/rss/How to Parse and Combine RSS News headlines using feedparser.ipynb @@ -15,7 +15,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 39, "metadata": { "collapsed": false }, @@ -40,7 +40,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 46, "metadata": { "collapsed": false, "scrolled": false @@ -50,81 +50,32 @@ "name": "stdout", "output_type": "stream", "text": [ - "Trump lashes out at Schiff over Russia probe memo\n", - "Former sports doctor sentenced to 40 to 125 years in prison\n", - "AP Exclusive: 2015 letter belies pope's claim of ignorance\n", - "Foles outduels Brady to give Eagles their first Super Bowl\n", - "Philadelphia cleaning up after some celebrations turn unruly\n", - "How the Eagles beat the Patriots at own game, and why the upset was historic\n", - "Who is Adam Schiff? Top Democrat Earns Nickname 'Little' From President Trump\n", - "New bipartisan immigration plan to be introduced in the Senate\n", - "Martin Luther King Jr. Commercial for Ram Trucks Is Swiftly Criticized\n", - "Justin Timberlake Halftime Selfie Kid Speaks: 'I Just Went for It'\n", - "Super Bowl anti-terrorism documents found on plane\n", - "Larry Nassar sentenced to 40 to 125 years in Eaton County\n", - "The Kentucky State Police's poorly received Super Bowl joke about jail rape\n", - "The first 'Solo: A Star Wars Story' trailer is here. But is it any good?\n", - "Sole Surviving Suspect in Paris Attacks Stands Trial in Belgium\n", - "Esmond Bradley Martin: Ivory investigator killed in Kenya\n", - "This is the fourth fatal crash involving an Amtrak train in two months\n", - "Everything you need to know about the Falcon Heavy launch\n", - "How the Ending of The Cloverfield Paradox Relates to the Other Cloverfield Movies\n", - "Rain put a damper on Super Bowl Sunday\n", - "These are the 3 best Amazon deals that you can get right now\n", - "Broadcom sweetens its bid for Qualcomm, calling it the 'best and final' offer\n", - "Israeli man stabbed to death at West Bank settlement\n", - "Autistic UK Man Accused of Hacking FBI Wins Appeal Against Extradition to US\n", - "Review: 'This Is Us' (thankfully) resists overdoing Jack's death\n", - "White House Plans To Withdraw 'Conspiracy Theorist And Anti-Science Extremist’ Pick\n", - "Hype over Republican Memo leads to calls for Justice Department firing\n", - "A Suspected Serial Killer May Have Targeted Toronto's Gay Village For Years\n", - "Oil tanker with 22 Indian crew missing in Gulf of Guinea since Friday\n", - "We're Never Going To 'Win' In Afghanistan\n", - "Lasers on planes used to reveal massive massive complex of Mayan ruins\n", - "Egypt says 4,400-year-old tomb discovered outside Cairo\n", - "Nassar Reportedly Abused Over 2 Dozen Girls And Women During Sluggish FBI Investigation\n", - "Alaskan copper mine proposal sparks concern about wild salmon fishery\n", - "Donald Trump will trigger constitutional crisis if he uses declassified memo to end Russia probe, Democrats warn\n", - "North Korea makes $200m by 'flouting' sanctions\n", - "Police release 'kidnapped' priest in DR Congo\n", - "Court hands Vietnam oil official another life sentence for corruption\n", - "Paul Ryan: Secretary Getting $1.50 More A Week Shows Effect Of GOP Tax Cuts\n", - "An Actual Nazi Is About To Get The Republican Nomination For A Congressional Seat\n", - "Hawaii man says he's devastated about sending missile alert\n", - "Concussions and Protests: Football's popularity drops\n", - "Man Charged With Selling Armor-Piercing Bullets to Las Vegas Shooter Stephen Paddock\n", - "Years Of U.S. Government Lies Could Soon Result In A Kurdish Massacre\n", - "Mexico: 300 migrants found in dangerously cramped trucks\n", - "Not just boy and girl; more teens identify as transgender\n", - "Congo rebel leader extradited from Tanzania to face trial\n", - "Father Of Otto Warmbier Will Attend Winter Olympics In South Korea: Report\n", - "Europe must brake mounting nuclear arms race: Germany\n", - "Leon Panetta on fallout from the Nunes memo\n", - "This Kid Looking At His Phone Is The Super Bowl's Best Meme\n", - "Larry Nassar: Thousands raised for father who charged at paedophile doctor jailed for abusing his gymnast daughters\n", - "Dreamers Need More Cities And States Ready To Defy Trump\n", - "Warren Buffett on hand as Navy commissions newest warship\n", - "A Controversial Bill Would Allow Chemical Castration of Sex Offenders in Oklahoma\n", - "Israel legalizes West Bank outpost after settler killed\n", - "Uma Thurman Says Harvey Weinstein Assaulted Her\n", - "Thousands of Greeks protest against Macedonia name compromise\n", - "Syrian rebels down Russian plane, kill pilot\n", - "These Pups Rescued From Puerto Rico Are Part Of This Year's Puppy Bowl\n", - "Woman who cut baby from pregnant neighbour's womb weeps as she apologises in court: 'There is no excuse. There is nothing'\n", - "Mike Pence to stop North Korea 'hijacking' Winter Olympics, aide says\n", - "Man dies after rescuing 9-year-old son from aqueduct in Hesperia\n", - "School system's appeals process leaves some minorities out\n", - "Illinois GOP Rep. Under Fire For Ad Mocking Transgender Community, Feminists\n", - "After An NFL Season Defined By Black Protest, The Super Bowl Sticks To Sports\n", - "U.S. forces begin reducing numbers in Iraq: Iraqi spokesman\n", - "Poland's top politician: Holocaust bill is 'misunderstood'\n", - "Bodies of around 20 migrants recovered from sea: Spanish official\n", - "Suspect arrested in Italy after gun attack on foreigners\n", - "Madeleine Albright Would Give Devin Nunes An 'F' For Memo Stunt\n", - "I tried living like Tom Brady for a week\n", - "Survey: Most residents in struggling US areas respect police\n", - "In Super Bowl ads, a play for values and a contentious MLK message\n", - "Indianapolis Colts' Edwin Jackson Killed By Suspected Drunk Driver\n" + "France bombs Islamic State HQ, hunts attacker who got away\n", + "Clinton campaign defends debate 9/11 remarks\n", + "Donald Trump: ‘O’Malley is a clown’; ‘Hillary is owned by Wall Street’\n", + "College student from California studying abroad killed in Paris attacks\n", + "About 1,500 Mormons resign from church in protest of same-sex policy\n", + "Pentagon says five Guantanamo detainees transferred to United Arab Emirates\n", + "Paris attacks: Video shows firefight outside Bataclan\n", + "Paris attacks show U.S. surveillance of Islamic State may be ‘going dark’\n", + "Belgian connection: At least 3 held in Brussels over Paris attacks\n", + "Clinton wobbled on foreign policy in debate\n", + "Clinton cites 9/11 in defending Wall Street donations\n", + "Sanders scores applause for Eisenhower quip\n", + "Sanders campaign claims victory in CBS dispute\n", + "Paris attacks may lead to US military anti-IS escalation\n", + "Sanders aide pushes back against CBS switch to foreign policy focus for debate\n", + "Five questions about Paris for Clinton, Sanders, and O’Malley\n", + "France Strikes ISIS Targets in Syria in Retaliation for Attacks - New York Times\n", + "Clinton's debate performance leaves trail of fodder for political adversaries - Washington Post\n", + "Parisians united: What the attacks mean to us - CNN\n", + "In wake of attacks, presidential contenders focus on Syrian refugees - Miami Herald\n", + "The Belgian neighborhood indelibly linked to jihad - Washington Post\n", + "Americans pay respects to Paris terror victims - USA TODAY\n", + "Pentagon transfers 5 Yemenis being held at Guantanamo Bay to UAE - Washington Post\n", + "Paris unites in defiant solidarity, then scatters in panic - Washington Post\n", + "Paris terror attack: Names of victims start to emerge - CNN\n", + "Patriots still undefeated after late field goal - NFL.com\n" ] } ], @@ -134,8 +85,8 @@ "\n", "# List of RSS feeds that we will fetch and combine\n", "newsurls = {\n", - " 'apnews': 'http://hosted2.ap.org/atom/APDEFAULT/3d281c11a96b4ad082fe88aa0db04305',\n", - " 'googlenews': 'https://news.google.com/news/rss/?hl=en&ned=us&gl=US',\n", + " 'apnews': 'http://hosted2.ap.org/atom/APDEFAULT/3d281c11a76b4ad082fe88aa0db04909',\n", + " 'googlenews': 'http://news.google.com/?output=rss',\n", " 'yahoonews': 'http://news.yahoo.com/rss/'\n", "}\n", "\n", @@ -180,9 +131,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.4" + "version": "3.5.0" } }, "nbformat": 4, - "nbformat_minor": 2 + "nbformat_minor": 0 }