Programming : Get All the Url on a Web Page - VB.net

I had tried this function before it work to me and help me a lot on my work in Programming. If you need this function, you just need to change the REGEXPATTERN and copy paste it to visual studio 2005(framework 2.0)

REGEXPATTERN - Replace Check URL Regex to REGEXPATTERN ( If you need to get Email Address from a Web Page, you can change it to Email checking Regex)

Imports System
Imports System.Text
Imports System.Collections.Generic
Imports System.Net
Imports System.IO
Imports System.Text.RegularExpressions

Partial Class GetAllUrlOnaPage
Inherits System.Web.UI.Page

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
Dim urls As New GetAllUrlOnaPage
GetWebUrls(RetrieveWeb(urls.ResolveClientUrl("http://www.google.com")))
End Sub

'get the content of the web page passed in
Private Function RetrieveWeb(ByVal webPage As String) As String
Dim response As HttpWebResponse = Nothing
Dim respStream As StreamReader = Nothing
Try
Dim request As HttpWebRequest = CType(WebRequest.Create(webPage), HttpWebRequest)
request.Timeout = 10000
response = CType(request.GetResponse, HttpWebResponse)
respStream = New StreamReader(response.GetResponseStream)
Return respStream.ReadToEnd
Catch ex As Exception
Throw ex
Finally
response.Close()
respStream.Close()
End Try
End Function

Sub GetWebUrls(ByVal content As String)
Dim pattern As String= "REGEXPATTERN"

Dim RegExpr As Regex = New Regex(pattern, RegexOptions.IgnoreCase)
Dim match As Match = RegExpr.Match(Content)
While match.Success
ResponseOut(match.Groups(0).Value, match.Groups(1).Value)
match = match.NextMatch

End While
End Sub

Private Sub ResponseOut(ByVal file As String, ByVal message As String)

Response.Write(file & " --- " + message + "

")
End Sub

Explore .NET Framework

The Microsoft .NET Framework is a software component that can be added to or is included with the Microsoft Windows operating system. It provides a large body of pre-coded solutions to common program requirements, and manages the execution of programs written specifically for the framework. The .NET Framework is a key Microsoft offering, and is intended to be used by most new applications created for the Windows platform.

The pre-coded solutions that form the framework's class library cover a large range of programming needs in areas including: user interface, data access, database connectivity, cryptography, web application development, numeric algorithms, and network communications. The functions of the class library are used by programmers who combine them with their own code to produce applications.

Programs written for the .NET Framework execute in a software environment that manages the program's runtime requirements. This runtime environment, which is also a part of the .NET Framework, is known as the Common Language Runtime (CLR). The CLR provides the appearance of an application virtual machine, so that programmers need not consider the capabilities of the specific CPU that will execute the program. The CLR also provides other important services such as security mechanisms, memory management, and exception handling. The class library and the CLR together compose the .NET Framework. The framework is intended to make it easier to develop computer applications and to reduce the vulnerability of applications and computers to security threats.

First released in 2002, it is included with Windows Server 2003 and Windows Vista, and can be installed on most older versions of Windows.

What is ASP.Net

ASP.NET is a web application framework marketed by Microsoft. Programmers can use it to build dynamic web sites, web applications and XML web services. It is part of Microsoft's .NET platform and is the successor to Microsoft's Active Server Pages (ASP) technology.

ASP.NET is built on the Common Language Runtime, meaning programmers can write ASP.NET code using any Microsoft .NET language.

ASPX file format
ASPX is a text file format used to create Webform pages; in programming jargon, the ASPX file typically contains static HTML or XHTML markup, as well as markup defining Web Controls and Web User Controls where the developer places all the required static and dynamic content for the web page. Additionally, dynamic code which runs on the server can be placed in a page within a block which is similar to other web development technologies such as PHP, JSP, and ASP, but this practice is generally discouraged except for Databinding.

The recommended method for dealing with dynamic program code is to use the code-behind model, which places this code in a separate file or in a specially designated script tag. Code-behind files are typically named something to the effect of MyPage.aspx.cs or MyPage.aspx.vb based on the ASPX file name (this practice is automatic in Microsoft Visual Studio and other IDEs). When using this style of programming, the developer writes code to respond to different events, like the page being loaded, or a control being clicked, rather than a procedural walk through the document

Other file extensions associated with different versions of ASP.NET include:
asax - Global.asax, used for application-level logic and event handling
ascx - Web UserControls: custom controls to be placed onto web pages.
ashx - custom Http handlers
asmx - web service pages.
axd - when enabled in web.config requesting trace.axd outputs application-level tracing. Also used for the special webresource axd handler which allows control/component developers to package a component/control complete with images, script, css etc. for deployment in a single file (an 'assembly')
browser - browser capabilities files stored in XML format; introduced in version 3.0. ASP.NET 2 includes many of these by default, to support common web browsers. These specify which browsers have which capabilities, so that ASP.NET 2 can automatically customize and optimize its output accordingly. Special .browser files are available for free download to handle, for instance, the W3C Validator, so that it properly shows standards-compliant pages as being standards-compliant. Replaces the harder-to-use BrowserCaps section that was in machine.config and could be overridden in web.config in ASP.NET 1.x.
config - web.config is the only file in a specific Web application to use this extension by default (machine.config similarly affects the entire Web server and all applications on it), however ASP.NET provides facilities to create and consume other config files. These are stored in XML format, so as to allow configuration changes to be made with simplicity.
cs/vb - In ASP.NET 2 any cs/vb files placed inside the App_Code folder are dynamically compiled and available to the whole application.
master -Master Pages; introduced in version 2.0
sitemap - sitemap configuration files
skin - theme skin files.
resx - resource files for internationalization and localization. Resource files can be global (for e.g. messages) or "local" which means specific for a single aspx or ascx file


ASP.NET compared to ASP classic
ASP.NET attempts to simplify developers' transition from Windows application development to web development by offering the ability to build pages composed of controls similar to a Windows user interface. A web control, such as a button or label, functions in very much the same way as its Windows counterpart: code can assign its properties and respond to its events. Controls know how to render themselves: whereas Windows controls draw themselves to the screen, web controls produce segments of HTML and JavaScript which form part of the resulting page sent to the end-user's browser.

ASP.NET encourages the programmer to develop applications using an event-driven GUI paradigm (event-driven GUI model), rather than in conventional web-scripting environments like ASP and PHP. The framework attempts to combine existing technologies such as JavaScript with internal components like "ViewState" to bring persistent (inter-request) state to the inherently stateless web environment.

Other differences compared to ASP classic are:
- Compiled code means applications run faster with more design-time errors trapped at the development stage.
- Significantly improved run-time error handling, making use of exception handling using try-catch blocks.
- Similar metaphors to Windows applications such as controls and events, which make development of rich user interfaces, previously only found on the desktop, possible.
- An extensive set of controls and class libraries allows the rapid building of applications, plus user-defined controls allow commonly used templates, such as menus. Layout of these controls on a page is easier because most of it can be done visually in most editors.
- ASP.NET leverages the multi-language capabilities of the .NET CLR, allowing web pages to be coded in VB.NET, C#, J#, etc.
- Ability to cache the whole page or just parts of it to improve performance.
Ability to use the code-behind development model to separate business logic from presentation.
- If an ASP.NET application leaks memory, the ASP.NET runtime unloads the AppDomain hosting the erring application and reloads the application in a new AppDomain.
- Session state in ASP.NET can be saved in a SQL Server database or in a separate process running on the same machine as the web server or on a different machine. That way session values are not lost when the web server is reset or the ASP.NET worker process is recycled.
- Previous versions of ASP.NET (1.0 and 1.1) were criticized for their lack of standards compliance. The generated HTML and JavaScript sent to the client browser would not always validate against W3C/ECMA standards. In addition, the framework's browser detection feature sometimes incorrectly identified web browsers other than Microsoft's own Internet Explorer as "downlevel" and returned HTML/JavaScript to these clients that was crippled or broken. However, in version 2.0, all controls generate valid HTML 4.0, XHTML 1.0 (the default) or XHTML 1.1 output, depending on the site configuration. Detection of standards-compliant web browsers is more robust and support for Cascading Style Sheets is more extensive.

* Web Server Controls: These are the most powerful controls introduced by ASP.net for providing the UI for the web form. These controls are state managed controls and are WYSWYG (What You See What You Get) controls