Web Page Inspector

An API to instantly retrieve clean, structured data from any URL

Analyze a URL

Paste the URL to an article to analyze it

Or, try these sample articles:

How It Works

Web Page Inspector attempts to extract the title, full text, both a short and long summary, permalink, social share count, tags, sentiment, and more from a valid URL. In this demo we provie both a UI of the metadata, as well as the raw JSON response.

Tools Used

At the heart of this demo is a microservice called Mega Analyze URL, which is a wrapper for capturing the metadata, summary, social sentiment analysis, tags, count of social shares, images, and links for a URL. Check out the source code here.

The underlying algorithms are:

  • Analyze URL - takes a URL and returns several pieces of structured data
  • Summarizer - creates a summary of the most important points of the original document
  • Social Sentiment Analysis - assigns sentiment ratings of "positive", "negative" and "neutral"
  • Count Social Shares - returns the number of times that URL has been shared on various social media sites
  • Get Links - scrapes the page for all the links and returns them as URL strings
  • Get Image Links- return a list of all the image URLs
  • Auto-Tag URL - produces candidate tags using LDA

Takeaway

By creating a microservice that wraps other algorithms, a developer can quickly generate useful structured data in a single API call without ever having to write their own web scraper or manage servers. This makes it possible to automatically extract clean text and metadata from any URL.

Built For Developers

A simple, scalable API for machine intelligence

SAMPLE INPUT

import Algorithmia

input = "http://thenewstack.io/algorithmia-new-algorithm-economy/"
client = Algorithmia.client('API KEY')
algo = client.algo('outofstep/MegaAnalyzeURL/0.1.1')

print algo.pipe(input)
LEARN MORE

Join the thousands of developers already building intelligent apps

Get 10,000 additional credits when you sign up using the code "demos"

SIGN UP FOR FREE

Algorithms as a Microservice

Leverage an ever-growing library of more than 2,200 algorithmic microservices via an intuitive API. We provide the tools and manage the cloud infrastructure needed to run it at scale.

Learn more

Web Services for Business Logic

Instantly deploy your backend code as an API for public or private consumption. Every algorithm runs as it's own microservice, making each composable, interoperable, and secure.

Learn more

Hosted Trained Models

Have a trained machine learning or deep learning model? Turn it into a serverless microservices in minutes. We'll show you how to get started for free, and scale with ease.

Learn more