Develop Reference

How Can I Use Regex to Match All the Urls in This String?

I hope to use the following code to get all URLs from a string.
But I only the three URLs ,there are http://www.google.com, https://www.twitter.com and www.msn.com.
I hope I can get all URLs include bing.com in the result, how can I modifty the var expression = /(https?:\/\/(?:www\.| ... ?
function openURLs() {
let links = "http://www.google.com Hello https://www.twitter.com The www.msn.com World bing.com";
if (links) {
var expression = /(https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9]+\.[^\s]{2,}|www\.[a-zA-Z0-9]+\.[^\s]{2,})/gi;
var url_array = links.match(expression);
if (url_array != null) {
url_array.forEach((url) => {
urlOK = url.match(/^https?:/) ? url : '//' + url;
window.open(urlOK)
});
}
}
}

Going off of what you currently have, you can just append |[a-zA-Z0-9]+\.[^\s]{2,} to the end of your expression. The resulting line will look like this:
var expression = /(https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9]+\.[^\s]{2,}|www\.[a-zA-Z0-9]+\.[^\s]{2,})|[a-zA-Z0-9]+\.[^\s]{2,}/gi;
This could be cleaner, but it'll do what you're asking.
Edit:
If you're okay with something slightly more permissive that can pull the same URLs out, you can try this expression:
var expression = /(?:https?:\/\/)?(?:www\.)?[\w.-]+\.\S{2,}/gi;

A permissive regular expression may be the following:
var expression = /(https?:\/\/)?[a-zA-Z0-9]+\.[a-zA-Z0-9]+\S*/
This expression is simpler and easier to debug. Furthermore, it will match any website, including the ones with query params (example.com?param=value) or with no ASCII characters (example.com/你好).
Here you can see a test.
On the other hand, it will match things that aren't websites as soon as they contain a dot, so things like foo.bar will be matched. However, there is no reliable way to detect whether strings like foo.bar are actually websites.

Get base url from string with Regex and Javascript

I'm trying to get the base url from a string (So no window.location).
It needs to remove the trailing slash
It needs to be regex (No New URL)
It need to work with query parameters and anchor links
In other words all the following should return https://apple.com or https://www.apple.com for the last one.
https://apple.com?query=true&slash=false
https://apple.com#anchor=true&slash=false
http://www.apple.com/#anchor=true&slash=true&whatever=foo
These are just examples, urls can have different subdomains like https://shop.apple.co.uk/?query=foo should return https://shop.apple.co.uk - It could be any url like: https://foo.bar
The closer I got is with:
const baseUrl = url.replace(/^((\w+:)?\/\/[^\/]+\/?).*$/,'$1').replace(/\/$/, ""); // Base Path & Trailing slash
But this doesn't work with anchor links and queries which start right after the url without the / before
Any idea how I can get it to work on all cases?

You could add # and ? to your negated character class. You don't need .* because that will match until the end of the string.
For your example data, you could match:
^https?:\/\/[^#?\/]+
Regex demo
strings = [
"https://apple.com?query=true&slash=false",
"https://apple.com#anchor=true&slash=false",
"http://www.apple.com/#anchor=true&slash=true&whatever=foo",
"https://foo.bar/?q=true"
];
strings.forEach(s => {
console.log(s.match(/^https?:\/\/[^#?\/]+/)[0]);
})

You could use Web API's built-in URL for this. URL will also provide you with other parsed properties that are easy to get to, like the query string params, the protocol, etc.
Regex is a painful way to do something that the browser makes otherwise very simple.
I know that you asked about using regex, but in the event that you (or someone coming here in the future) really just cares about getting the information out and isn't committed to using regex, maybe this answer will help.
let one = "https://apple.com?query=true&slash=false"
let two = "https://apple.com#anchor=true&slash=false"
let three = "http://www.apple.com/#anchor=true&slash=true&whatever=foo"
let urlOne = new URL(one)
console.log(urlOne.origin)
let urlTwo = new URL(two)
console.log(urlTwo.origin)
let urlThree = new URL(three)
console.log(urlThree.origin)

const baseUrl = url.replace(/(.*:\/\/.*)[\?\/#].*/, '$1');

This will get you everything up to the .com part. You will have to append .com once you pull out the first part of the url.
^http.*?(?=\.com)
Or maybe you could do:
myUrl.Replace(/(#|\?|\/#).*$/, "")
To remove everything after the host name.

JavaScript replace words which aren't in a URL

I'm running a JavaScript which replaces certain words in my browser's text content.
However I do not wish for it to replace the words within url's.
UPDATE:
E.g., if I've replaced X with Y, and I search for X within a search engine, any url links with X in it are replaced with Y - I can't click on them as they don't exist (and/or they are incorrect).
document.body.innerHTML = document.body.innerHTML.replace(/word/gi, "newword");
How can I do this?

It's really hard to do this (I mean, its too broad), but I suggest you to do that in these few steps:
first you should match all urls and store them in some array (e.g. var urls = [];)
also replace then all urls with some unique characters sequence, which are not for sure in your browser's content (e.g. ~~~~~)
then do your clasical replace, like document.body.innerHTML = document.body.innerHTML.replace(/word/gi, "newword");
and finally match in your new replaced browser's content all yours specials characters sequence (~~~~~) and replace them back in the same order with urls stored in your array (urls).
Matching URLs:
About matching urls you need a good regex that matches urls. This is hard to do. See here, here and here:
...almost anything is a valid URL. There
are some punctuation rules for
splitting it up. Absent any
punctuation, you still have a valid
URL.
Check the RFC carefully and see if you
can construct an "invalid" URL. The
rules are very flexible.
For example ::::: is a valid URL.
The path is ":::::". A pretty
stupid filename, but a valid filename.
Also, ///// is a valid URL. The
netloc ("hostname") is "". The path
is "///". Again, stupid. Also
valid. This URL normalizes to "///"
which is the equivalent.
Something like "bad://///worse/////"
is perfectly valid. Dumb but valid.
Anyway, this answer is not meant to give you the best regex but rather a proof of how to do the string wrapping inside the text, with JavaScript.
OK so lets just use this one: /(https?:\/\/[^\s]+)/g
Again, this is a bad regex. It will have many false positives. However it's good enough for this example.
function urlify(text) {
var urlRegex = /(https?:\/\/[^\s]+)/g;
return text.replace(urlRegex, function(url) {
return '' + url + '';
})
// or alternatively
// return text.replace(urlRegex, '$1')
}
var text = "Find me at http://www.example.com and also at http://stackoverflow.com";
var html = urlify(text);
// html now looks like:
// "Find me at http://www.example.com and also at http://stackoverflow.com"
So in sum try:
$$('#pad dl dd').each(function(element) {
element.innerHTML = urlify(element.innerHTML);
});
I hope that it will do at least a little help for you.

Here is a simple solution:
1. Replace all "word"s in urls with a "tempuniqueflag" (Note that word is not a substring of tempuniqueflag)
var urls = document.querySelectorAll('a');
for (url in urls) {
if (typeof urls[url].href === "string")
urls[url].href = urls[url].href.replace(/word/,"tempuniqueflag");
}
Replace your text content as usual
document.body.innerHTML = document.body.innerHTML.replace(/word/gi, "newword");
Bring back the original word in the urls
for (url in urls) {
if (typeof urls[url].href === "string")
urls[url].href = urls[url].href.replace(/tempuniqueflag/,"word");
}

How to get the main domain string using regular expression?

I have just started using regular expression and i landed up in a problem. So it would be really nice if someone can help me out with it.
The problem is, in case I have a url as given below;
$url = http://www.blog.domain.com/page/category=?
and want only the domain, how can i get it using regular expression in javascript.
thank you

This should work too, but most restrictive and shorter:
var url = "http://www.blog.domain.com/page/category"
var result = url.replace(/^(https?:\/\/)?(.+\.)*(([a-z0-9-]*)\.[a-z]{2,6})(\/.+)$/i,"$4")
If you want "domain.com" and not only "domain", use $3 instead of $4.
Explaination step by step:
A correct domain syntax: letters,numbers and "-" /([a-z0-9-]*)/i
Domain extension (2-6 chars): /(([a-z0-9-]*)\.[a-z]{2,6})/i
Subdomains: /(.+\.)*(([a-z0-9-]*)\.[a-z]{2,6})/i
An url start with http and maybe https: /^https?:\/\/(.+\.)*(([a-z0-9-]*)\.[a-z]{2,6})/i
You can put or not http when you type an url: /^(https?:\/\/)?(.+\.)*(([a-z0-9-]*)\.[a-z]{2,6})/i
Then what is after /: /^(https?:\/\/)?(.+\.)*(([a-z0-9-]*)\.[a-z]{2,6})(\/.+)$/i

Try below code
var url = "http://www.blog.domain.com/page/category=?";
var match = url .match(/(?:http?:\/\/)?(?:www\.)?(.*?)\//);
console.log(match[match.length-1]);

You can get it using the following RegEx: /.*\.(.+)\.[com|org|gov]/
You can add all of the supported domain extensions in this regex.
RegEx101 Explanation
Working Code Snippet:
var url = "http://www.blog.domain.gov/page/category=?";
var regEx = /.*\.(.+)\.[com|org|gov]/;
alert(url.match(regEx)[1]);

Do not use regex for this:
use hostname:
The URLUtils.hostname property is a DOMString containing the domain of
the URL.
var x = new URL("http://www.blog.domain.com/page/category=?").hostname;
console.log(x);
as pointed by vishwanath, URL faces compatibilty issues with IE<10 so for those cases, regex will be needed.
use this :
var str = "http://www.blog.domain.com/page/category=?";
var res = str.match(/[^.]*.(com|net|org|info|coop|int|co\.uk|org\.uk|ac\.uk|uk)/g);
console.log(res);
=> domain.com
the list in the regex can be expanded further depending upon your need.
a list of TLDs can be found here

Simple Regex question

I want to remove this from a url string
http://.....?page=1
I know this doesn't work, but I was wondering how you would do this properly.
document.URL.replace("?page=[0-9]", "")
Thanks

It seems like you want to get rid of the protocol and the querystring. So how about just concatenating the remaining parts?
var loc = window.location;
var str = loc.host + loc.pathname + loc.hash;
http://jsfiddle.net/9Ng3Z/
I'm not entirely certain what the requirements are, but this fairly simple regex works.
loc.replace(/https?\:\/\/([^?]+)(\?|$)/,'$1');
It may be a naive implementation, but give it a try and see if it fits your need.
http://jsfiddle.net/9Ng3Z/1/

? is a regex special character. You need to escape it for a literal ?. Also use regular expression literals.
document.URL.replace(/\?page=[0-9]/, "")

The answer from #patrick dw is most practical but if you're really curious about a regular expression solution then here is what I would do:
var trimUrl = function(s) {
var r=/^http:\/\/(.*?)\?page=\d+.*$/, m=(""+s).match(r);
return (m) ? m[1] : s;
}
trimUrl('http://foo.com/?page=123'); // => "foo.com/"
trimUrl('http://foo.com:8080/bar/?page=123'); // => "foo.com:8080/bar/"
trimUrl('foobar'); // => "foobar"

You're super close. To grab the URL use location.href and make sure to escape the question mark.
var URL = location.href.replace("\?page=[0-9]", "");
location.href = URL; // and redirect if that's what you intend to do
You can also strip all query string parameters:
var URL = location.href.replace("\?.*", "");