Link Indexing techniques
Here are some key points one can index links based on:
1. find all links in a webpage and add them to the index
2. create a low-frequency hash of the URL and add it to the index
3. if two URLs contain same words, add both URLs to the index
4. crawl every page on the link path
you can use either one or a combination of the above techniques depending on your need for each use case scenario. so what should be next? how do we start? 1) once you have identified using any one of the techniques, grab HTML from that webpage which has links in it 2) extract out URLs from HTML 3) finally run through all these extracted URLs and perform indexing based upon applied strategy [eg: take a look at "shodan" at https://www.shodan.io/ ]
honeypots for dummies: hey, what's up? how are you today? do you feel like learning something new about honeypots? 1) sure 2) let me start with a quick intro to a particular type of security solution known as "intrusion detection systems".
these types of solutions capture and analyze network traffic in order to detect whether any malicious activity has been observed or not 3) so far so good... next question is, which type of IDS should I be using or where can I learn about this topic 4) no worries.. I have prepared a list of resources that will help you get started https://github.com/honeynet/docs-lab
this is a really good start. what should be the next step... well, i am glad you asked 5) let's pick up one of these resources and get started.. but many of them look similar.. which one do we choose 6) do not worry about it 7) pick any one that looks approachable to you https://github.com/honeynet/docs-lab
what do we got here? looking through these documents and not sure where to begin? I feel your pain 8) honeypots for dummies: this article on howstuffworks [https://www.howtogeek.com/133791/begin-geek-guide-to-honeypots/ ] simplifies everything about honeypots in a very digestible form for any one who is new to this field 9) honeypot, what is it? how does it work? an introduction for beginners 10) hmm.. that looks interesting. i am starting to get interested in this topic 11) next, let's look into the types of attacks covered by IDS -Buffer overflow
-Denial of service attack
-SQL injection attack
12) I have just started learning about honeypots and would like to learn more 13) you are pretty cool... if only there was some list where the field experts could share their knowledge with us 14) yeah i got your back 15). alright so now head over to [https://github.com/honeynet/document-specs] and you will find a document that looks like: link: https://github.com/honeynet/document-specs
title: Detection Types? feel free to browse through these documents and use it as a reference 16). next, we need to look into how honeypots work internally 17) hey i hear you 18) let's take a moment and understand what is happening when we set up a basic honeypot 19) basics of how honeypots works 20) now let's look at some more advanced stuff 21) here is an interesting paper on the same topic 22) alright so far so good.. looking good 23). i am still pretty fresh and would like to learn more about this subject 24) you're right, honeypots are pretty sweet.. now let's look into some of the toolkits that can be used for building your own honeypot
title: HoneyCloud - A versatile Platform as a Service (PaaS) for virtualized Honeypots
this document looks interesting 25). i hope we can use it as a reference 26) yup! simply click on the link and read through it 27) oh, there is one more thing i would like to mention 28) it's about an online platform that lets you try out different honeypot technologies in the cloud 29) here is a link for it 30) [https://honeycloud.io]
operational guidance for national tech service providers:
title: Operational Guidance for National Tech Service Providers - A pioneering study on the challenges and pitfalls of technical threat intelligence sharing throughout Europe 31) this looks like some sort of research paper 32) yes, but it touches upon many things that are relevant to honeypots 33) let me summarize it 34). According to one of the earliest studies into these problems, "many law enforcement agencies are neither able to provide timely nor specific threat information to the service providers". In addition, there are many integrative and/or technical challenges that must be overcome before such sharing can be considered reliable.
35) we should definitely investigate claims like this one: "The common complaint is that private investigators do not trust public agencies." 36) okay so it's time to get our hands dirty :) 37). you see honeypots work by posing as vulnerable services on an operating system 38). so if we set up a virtual machine with network interfaces and expose them or configure them in such a way that they look vulnerable, then attackers will get curious and try their luck 39) great! now let's install some toolkit 40).