What are some good tools for interactive web scraping and automation?
Let’s learn what are some good tools for interactive web scraping and automation. The most accurate or helpful solution is served by Quora.
There are ten answers to this question.
Best solution
Tools for automating or scripting web app access, including rich JavaScript apps. Commercial or open source. Clarification Looks like the answers tools are for classic "scraping" or parsing websites read-only. I'm looking for a solution for those cases where the user needs to load forms or even run JavaScript client-side functions.
Answer:
BeautifulSoup is a Python library for HTML and XML parsing. Great for extracting data from web!
Samarth Bansal at Quora Mark as irrelevant Undo
Other solutions
What are the best tools (free/paid) to scrape large amounts of data from webpages and store it in required formats? The tool preferably should be automated to look for new data at specific intervals.
Answer:
What you are looking for should be able to extract large amounts of websites super fast and should have...
Vicky Rathee at Quora Mark as irrelevant Undo
Answer:
Act-On Software and Pardot are best for smaller companies,. Marketo for midsize and Eloqua for very...
Jeff Ogden at Quora Mark as irrelevant Undo
Answer:
You may also want to take a look at Sikuli. It's less widespread than Selenium but still is a good candidate...
Artyom Silivonchik at Quora Mark as irrelevant Undo
How do I extract content from a website like OKCupid for personal use? I basically want to spider the site to extract info from matches that'll dump into a table. This is purely for my own personal nerdy fun (ie doing some sort of clustering on the matches...
Answer:
If you're going to do your own clustering and write custom search algorithms, I'll assume you're a competent...
ribboncake at Ask.Metafilter.Com Mark as irrelevant Undo
Ones that can parse millions of web pages in a reasonable amount of time. Some of the desired features would be multi-threaded HTTP/1.1 persistent connections, distributed capabilities, ease of use, XPath or similar.
Answer:
Fastest scraping tools can be developed under JAVA environment by implementing proxy management (Multiple...
Heena Desai at Quora Mark as irrelevant Undo
I would like to build a proxy to use a remote JavaScript website through a REST API, to be accessed from a website or a mobile app. An example would a single-page JavaScript website or app with client-side logic that needs to be executed before calling...
Nidal AlHariri at Quora Mark as irrelevant Undo
I want to collect data from a few sites and have it dumped into Excel in a semi-organized way. What tools exists for me to do this easily?
Answer:
Unfortunately, there isn't really a "point and click" web scraping tool. If the data you want...
Ashish Datta at Quora Mark as irrelevant Undo
I have an idea for a project that would require the ability to search a dictionary of words and find the year of it's known introduction (as close as possible).I am aware of etymology-online (love that site), but since, as far as I'm aware, it's just...
Answer:
Ask Before. And i recommend python for scrapping. Maybe you can ask on opendata on reddit and stack...
symbioid at Ask.Metafilter.Com Mark as irrelevant Undo
What are the newest web marketing strategies that will impact all internet marketing professionals in the coming years ? A few to start with : Marketing Automation Retargeting Behavioral targeting Individual tracking & cohorts Conditionnal emailing...
Answer:
Here is an infographic that illustrates the key trends that marketers need to focus on in 2013. Please...
Christopher Fredrick at Quora Mark as irrelevant Undo
Related Q & A:
- What is the best language for HTML parsing and web scraping?Best solution by Quora
- What is the best web scraping software for building contact information databases from online directories?Best solution by Quora
- What web scraping tool is the best to extract data?Best solution by Quora
- What is a good web page to apply for a job?Best solution by Yahoo! Answers
- What are some good web design trends for 2010?Best solution by Quora
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.