intelmq.bots.parsers.shodan package

Submodules

intelmq.bots.parsers.shodan.parser module

Shodan Stream Parser

Copyright (C) 2018 by nic.at GmbH

intelmq.bots.parsers.shodan.parser.BOT

alias of intelmq.bots.parsers.shodan.parser.ShodanParserBot

exception intelmq.bots.parsers.shodan.parser.NoValueException(msg: typing.Union[str, NoneType] = None) → None

Bases: Exception

Raised in a conversion function in case the value cannot be used, e.g when trying to get the first item of an empty list

class intelmq.bots.parsers.shodan.parser.ShodanParserBot(bot_id: str, start: bool = False, sighup_event=None, disable_multithreading: bool = None)

Bases: intelmq.lib.bot.ParserBot

Parse Shodan data collected via the Shodan API

apply_mapping(mapping: typing.Dict[str, typing.Any], data: typing.Dict[str, typing.Any], key_path: typing.Tuple[str, ...] = ()) → typing.Dict[str, typing.Any]
ignore_errors = True
minimal_mode = False
process() → None
intelmq.bots.parsers.shodan.parser._dict_dict_to_obj_list(x: typing.Dict[str, typing.Dict[str, typing.Any]], identifier: str = 'identifier') → typing.List[typing.Dict[str, typing.Any]]

convert e.g {‘OuterKey1’: {‘InnerKey1’: ‘Value1’}, ‘OuterKey2’: {‘InnerKey2’: ‘Value2’}} to [{‘identifier’: ‘OuterKey1’, ‘InnerKey’: ‘Value1}, {‘identifier’: ‘OuterKey2’, ‘InnerKey’: ‘Value2’}}]

intelmq.bots.parsers.shodan.parser._get_first(variable: typing.List[typing.Any]) → typing.Any

get first element from list, if the list has any; raise NoValueException otherwise

intelmq.bots.parsers.shodan.parser._get_first_fqdn(variable: typing.List[str]) → str

get first valid FQDN from a list of strings

intelmq.bots.parsers.shodan.parser._keys_conversion(x: typing.Dict[str, typing.Any]) → typing.List[str]

extracts object keys to a list, for cases where the values they map to are empty/irrelevant

intelmq.bots.parsers.shodan.parser._maybe_single_to_list(x: typing.Any) → typing.List[typing.Any]

converts non-list objects to lists with a single item and leaves lists as-is, used to harmonize fields which avoid lists when a single value is given

Module contents