Web scraping with Azure Functions and Python

[ad_1]

Azure Functions are – comparable to AWS Lambda or Google Cloud Functions – event-based serverless functions that can be executed as required. The trigger for an Azure Function can be an HTTP request for the web API interface. An incoming IoT stream, a database change or execution at a specified time can also be used as triggers.

Depending on the requirements, the provider of Azure Functions (Microsoft) provides the underlying resources of the cloud infrastructure dynamically and scales them proportionally to demand. With this dynamic, demand-driven provision of the serverless functions, users only pay for the number of executions, the execution time and the memory consumption in the consumption plan (Hosting). The practical example at the end of the article shows the costs that can arise.

When using serverless functions, developers benefit from the fact that they no longer have to worry about the infrastructure, i.e. the provision and maintenance of servers – instead, they can concentrate fully on the development process of their application. However, you should be careful not to make the code too complex so that the serverless function can react quickly to the trigger or the triggering event at any time.

As a rule, Azure Functions are used in conjunction with other cloud services. However, they are also recommended in two exemplary scenarios in solo use:

  • Time-controlled web scraping and persistence of information in a cloud database such as Cosmos DB for later processing or presentation on a dashboard (see Fig. 1)
  • Web API of an AI model (Azure Function triggered by HTTP request)

Timed web scraping and persistence of information in Cosmos DB.

The Azure documentation describes in detail creating an Azure Function in the Command Line Interface (CLI) and in Visual Studio Code. This article takes up the documentation and shows how an Azure function can be expanded into a web scraper with the HTTP trigger in a few simple steps. Then it is connected to Cosmos DB in order to then make it available as a time-controlled Azure function in the cloud. There she automatically collects information from the Internet on a daily basis.

To home page

[ad_2]

Share:

Facebook
Twitter
Pinterest
LinkedIn

Leave a Reply

Related Posts

Can You Snort Marijuana

Can You Snort Marijuana Simply put, snorting marijuana is not a smart or safe way to use it. Most people smoke, vape, or eat edibles

Can Marijuana Cause Diarrhea

Can Marijuana Cause Diarrhea Simply put, using marijuana can sometimes cause stomach problems like diarrhea, but it’s not completely clear if one directly causes the

Drying Marijuana in Paper Bags

Drying Marijuana in Paper Bags Key Takeaways Discover why using paper bags is advantageous for drying marijuana Learn essential steps and expert techniques for efficiently

Can You Vacuum Seal Marijuana

Can You Vacuum Seal Marijuana You can totally vacuum seal your weed, no problem. Vacuum sealing helps keep your stash fresh and potent by getting