How can I extract only top-level and second-level domain from a URL using regex? I want to skip all lower level domains. Any ideas?
Here's my idea,
Match anything that isn't a dot, three times, from the end of the line using the $ anchor.
The last match from the end of the string should be optional to allow for .com.au or .co.nz type of domains.
Both the last and second last matches will only match 2-3 characters, so that it doesn't confuse it with a second-level domain name.
Regex:
[^.]*\.[^.]{2,3}(?:\.[^.]{2,3})?$
Demonstration:
Regex101 Example
Updated 2019
This is an old question, and the challenge here is a lot more complicated as we start adding new vanity TLDs and more ccTLD second level domains (e.g. .co.uk, .org.uk). So much so, that a regular expression is almost guaranteed to return false positives or negatives.
The only way to reliably get the primary host is to call out to a service that knows about them, like the Public Suffix List.
There are several open-source libraries out there that you can use, like psl, or you can write your own.
Usage for psl is quite intuitive. From their docs:
var psl = require('psl');
// Parse domain without subdomain
var parsed = psl.parse('google.com');
console.log(parsed.tld); // 'com'
console.log(parsed.sld); // 'google'
console.log(parsed.domain); // 'google.com'
console.log(parsed.subdomain); // null
// Parse domain with subdomain
var parsed = psl.parse('www.google.com');
console.log(parsed.tld); // 'com'
console.log(parsed.sld); // 'google'
console.log(parsed.domain); // 'google.com'
console.log(parsed.subdomain); // 'www'
// Parse domain with nested subdomains
var parsed = psl.parse('a.b.c.d.foo.com');
console.log(parsed.tld); // 'com'
console.log(parsed.sld); // 'foo'
console.log(parsed.domain); // 'foo.com'
console.log(parsed.subdomain); // 'a.b.c.d'
Old answer
You could use this:
(\w+\.\w+)$
Without more details (a sample file, the language you're using), it's hard to discern exactly whether this will work.
Example: http://regex101.com/r/wD8eP2
Also, you can likely do that with some expression similar to,
^(?:https?:\/\/)(?:w{3}\.)?.*?([^.\r\n\/]+\.)([^.\r\n\/]+\.[^.\r\n\/]{2,6}(?:\.[^.\r\n\/]{2,6})?).*$
and add as much as capturing groups that you want to capture the components of a URL.
Demo
If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.
RegEx Circuit
jex.im visualizes regular expressions:
For anyone using JavaScript and wanting a simple way to extract the top and second level domains, I ended up doing this:
'example.aus.com'.match(/\.\w{2,3}\b/g).join('')
This matches anything with a period followed by two or three characters and then a word boundary.
Here's some example outputs:
'example.aus.com' // .aus.com
'example.austin.com' // .austin.com
'example.aus.com/howdy' // .aus.com
'example.co.uk/howdy' // .co.uk
Some people might need something a bit cleverer, but this was enough for me with my particular dataset.
Edit
I've realised there are actually quite a few second-level domains which are longer than 3 characters (and allowed). So, again for simplicity, I just removed the character counting element of my regex:
'example.aus.com'.match(/\.\w*\b/g).join('')
Since TLDs now include things with more than three-characters like .wang and .travel, here's a regex that satisfies these new TLDs:
([^.\s]+\.[^.\s]+)$
Strategy: starting at the end of the string, look for one or more characters that aren't periods or whitespace, followed by a single period, followed by one or more characters that aren't periods or whitespace.
http://regexr.com/3bmb3
With capturing groups you can achieve some magix.
For example, consider the following javascript:
let hostname = 'test.something.else.be';
let domain = hostname.replace(/^.+\.([^\.]+\.[^\.]+)$/, '$1');
document.write(domain);
This will result in a string containing 'else.com'. This is because the regex itself will match the complete string and the capturing group will be mapped to $1. So it replaces the complete string 'test.something.else.com' with '$1' which is actually 'else.com'.
The regex isn't pretty and can probably be made more dynamic with things like {3} for defining how many levels deep you want to look for subdomains, but this is just an illustration.
if you want all specific Top Level Domain name then you can write regular expression like this:
[RegularExpression("^(https?:\\/\\/)?(([\\w]+)?\\.?(\\w+\\.((za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw))))\\/?$", ErrorMessage = "Is not a valid fully-qualified URL.")]
You can also put more domain name from this link:
https://www.icann.org/resources/pages/tlds-2012-02-25-en
The following regex matches a domain with root and tld extractions (named capture groups) from a url or domain string:
(?:\w+:\/{2})?(?<cs_domain>(?<cs_domain_sub>(?:[\w\-]+\.)*?)(?<cs_domain_root>[\w\-]+(?<cs_domain_tld>(?:\.\w{2})?(?:\.\w{2,3}|\.xn-+\w+|\.site|\.club))))\|
It's hard to say if it is perfect, but it works on all the test data sets that I have put it against including .club, .xn-1234, .co.uk, and other odd endings. And it does it in 5556 steps against 40k chars of logs, so the efficiency seems reasonable too.
If you need to be more specific:
/\.(?:nl|se|no|es|milru|fr|es|uk|ca|de|jp|au|us|ch|it|io|org|com|net|int|edu|mil|arpa)/
Based on http://www.seobythesea.com/2006/01/googles-most-popular-and-least-popular-top-level-domains/
Related
I'm building an app on NodeJS that uses Figma API, and I need to check if the string passed by a user is a valid Figma link. I'm currently using this simple regex expression to check the string:
/^https\:\/\/www.figma.com\/.*/i
However, it matches all links from figma.com, even the home page, not only links to the files and prototypes. Here is an example Figma link that should match:
https://www.figma.com/file/OoYmkiTlusAzIjYwAgSbv8wy/Test-File?node-id=0%3A1
Also the match should be positive if this is a prototype link, with proto instead of file in the path.
Moreover, since I'm using the Figma API, it would be useful to extract necessary parts of the URL such as the file ID and node ID at the same time.
TL;DR
✅ Use this expression to capture four most important groups (type, file id, file name and URL properties) and work from there.
/^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/?([^\?]+)?(.*))?$/
From the docs
This is the regex expression code provided by Figma on their developer documentation page about embeds:
/https://([w.-]+.)?figma.com/(file|proto)/([0-9a-zA-Z]{22,128})(?:/.*)?$/
🛑 However, it doesn't work in JS as the documentation is currently wrong and this expression has multiple issues:
Slashes and a dots are not escaped with backslashes.
It doesn't match from the start of the string. I added the start of string anchor ^ after VLAZ pointed it out in the comments. This way we will avoid matching strings that don't start with https, for example malicious.site/?link=https://figma.com/...
It will match not only www. subdomain but any other amount of W which is not great (e.g. wwwww.) — it can be fixed by replacing letter match with a simpler expression. Also this is a useless capturing group, I'll make it non-capturing.
It would be nice if the link matched even if it doesn't begin with https:// as some engines (e.g. Twitter) strip this part for brevity and if person is copying a link from there, it should still be valid.
After applying all the improvements, we are left with the following expression:
/^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/.*)?$/
There is also a dedicated NPM package that simply checks the URL against the similar pattern. However, it contains some of the flaws listed above so I don't advice using it, especially for just one line of code.
Extracting parts of the URL
This expression is extremely useful to use with Figma API as it even extracts necessary parts from the URL such as type of link (proto/file) and the file key. You can access them by indexes.
You can also add a piece of regex to match specific keys in the query such as node-id:
/^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/.*)?node-id=([^&]*)$/
Now you can use it in code and get all the parts of the URL separately:
var pattern = /^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/.*)?node-id=([^&]*)$/
var matched = 'https://www.figma.com/file/OoYmkiTlusAzIjYwAgSbv8wy/Test-File?node-id=0%3A1'.match(pattern)
console.log('url:', matched[0]) // whole matched string
console.log('type:', matched[1]) // group 1
console.log('file key:', matched[2]) // group 2
console.log('node id:', matched[3]) // group 3
Digging deeper
I spent some time recreating this expression almost from scratch so it would match as many possible Figma file/prototype URLs without breaking things. Here are three similar versions of it that would work for different cases.
✅ This version captures the URL parameters and the name of the file separately for easier processing. You can check it here. I added it in the beginning of the answer, because I think it's the cleanest and most useful solution.
/^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/?([^\?]+)?(.*))?$/
The groups in it are as following:
Group 1: file/proto
Group 2: file key/id
Group 3: file name (optional)
Group 4: url parameters (optional)
✅ Next up, I wanted to do the same but separating the /duplicate part that can be added in the end of any Figma URL to create a duplicate of the file upon opening.
/^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/?([^\?]+)?([^\/]*)(\/duplicate)?)?$/
✅ And back to the node-id parameter. The following regex expression finds and captures multiple URLs inside a multiline string successfully. The only downside that I found in the end is that it (as well as all the previous ones) doesn't check if this URL contains unencoded special characters meaning that it can potentially break things, but it can be avoided by manually encoding all parameters using encodeURI() function.
/^(?:https:\/\/)?(?:www\.)?figma\.com\/(file|proto)\/([0-9a-zA-Z]{22,128})(?:\/([^\?\n\r\/]+)?((?:\?[^\/]*?node-id=([^&\n\r\/]+))?[^\/]*?)(\/duplicate)?)?$/gm
There are six groups that can be captured by this expression:
Group 1: file/proto
Group 2: file key/id
Group 3: file name (optional)
Group 4: url parameters (optional)
Group 5: node-id (optional; only present when group 4 is present)
Group 6: /duplicate
And, finally, here is the example of a match and its groups (or try it yourself):
I want to match only parent domain name from an email address, which might or might not have a subdomain.
So far I have tried this:
new RegExp(/.+#(:?.+\..+)/);
The results:
Input: abc#subdomain.maindomain.com
Output: ["abc#subdomain.domain.com", "subdomain.maindomain.com"]
Input: abc#maindomain.com
Output: ["abc#maindomain.com", "maindomain.com"]
I am interested in the second match (the group).
My objective is that in both cases, I want the group to match and give me only maindomain.com
Note: before the down vote, please note that neither have I been able to use existing answers, nor the question matches existing ones.
One simple regex you can use to get only the last 2 parts of the domain name is
/[^.]+\.[^.]$/
It matches a sequence of non-period characters, followed by period and another sequence of non-periods, all at the end of the string. This regex doesn't ensure that this domain name happens after a "#". If you want to make a regex that also does that, you could use lazy matching with "*?":
/#.*?([^.]+\.[^.])$/
However,I think that trying to do everything at once tends to make the make regexes more complicated and hard to read. In this problem I would prefer to do things in two steps: First check that the email has an "#" in it. Then you get the part after the "#" and pass it to the simple regex, which will extract the domain name.
One advantage of separating things is that some changes are easier. For example, if you want to make sure that your email only has a single "#" in it its very easy to do in a separate step but would be tricky to achieve in the "do everything" regex.
You can use this regex:
/#(?:[^.\s]+\.)*([^.\s]+\.[^.\s]+)$/gm
Use captured group #1 for your result.
It matches # followed by 0 or more instance of non-DOT text and a DOT i.e. (?:[^.\s]+\.)*.
Using ([^.\s]+\.[^.\s]+)$ it is matching and capturing last 2 components separated by a DOT.
RegEx Demo
With the following maindomain should always return the maindomain.com bit of the string.
var pattern = new RegExp(/(?:[\.#])(\w[\w-]*\w\.\w*)$/);
var str = "abc#subdomain.maindomain.com";
var maindomain = str.match(pattern)[1];
http://codepen.io/anon/pen/RRvWkr
EDIT: tweaked to disallow domains starting with a hyphen i.e - '-yahoo.com'
I have been trying to make a Reg Exp to match the URL with specific domain name.
So if i want to check if this url is from example.com
what reg exp should be the best?
This reg exp should match following type of URLs:
http://api.example.com/...
http://preview.example.com/...
http://www.example.com/...
http://purhcase.example.com/...
Just simple rule, like http://{something}.example.com/{something} then should pass.
Thank you.
I think this is what you're looking for: (https?:\/\/(.+?\.)?example\.com(\/[A-Za-z0-9\-\._~:\/\?#\[\]#!$&'\(\)\*\+,;\=]*)?).
It breaks down as follows:
https?:\/\/ to match http:// or https:// (you didn't mention https, but it seemed like a good idea).
(.+?\.)? to match anything before the first dot (I made it optional so that, for example, http://example.com/ would be found
example\.com (example.com, of course);
(\/[A-Za-z0-9\-\._~:\/\?#\[\]#!$&'\(\)\*\+,;\=]*)?): a slash followed by every acceptable character in a URL; I made this optional so that http://example.com (without the final slash) would be found.
Example: https://regex101.com/r/kT8lP2/1
Use indexOf javascript API. :)
var url = 'http://api.example.com/api/url';
var testUrl = 'example.com';
if(url.indexOf(testUrl) !== -1) {
console.log('URL passed the test');
} else{
console.log('URL failed the test');
}
EDIT:
Why use indexOf instead of Regular Expression.
You see, what you have here for matching is a simple string (example.com) not a pattern. If you have a fixed string, then no need to introduce semantic complexity by checking for patterns.
Regular expressions are best suited for deciding if patterns are matched.
For example, if your requirement was something like the domain name should start with ex end with le and between start and end, it should contain alphanumeric characters out of which 4 characters must be upper case. This is the usecase where regular expression would prove beneficial.
You have simple problem so it's unnecessary to employ army of 1000 angels to convince someone who loves you. ;)
Use this:
/^[a-zA-Z0-9_.+-]+#(?:(?:[a-zA-Z0-9-]+\.)?[a-zA-Z]+\.)?
(domain|domain2)\.com$/g
To match the specific domain of your choice.
If you want to match only one domain then remove |domain2 from (domain|domain2) portion.
It will help you. https://www.regextester.com/94044
Not sure if this would work for your case, but it would probably be better to rely on the built in URL parser vs. using a regex.
var url = document.createElement('a');
url.href = "http://www.example.com/thing";
You can then call those values using the given to you by the API
url.protocol // (http:)
url.host // (www.example.com)
url.pathname // (/thing)
If that doesn't help you, something like this could work, but is likely too brittle:
var url = "http://www.example.com/thing";
var matches = url.match(/:\/\/(.[^\/]+)(.*)/);
// matches would return something like
// ["://example.com/thing", "example.com", "/thing"]
These posts could also help:
https://stackoverflow.com/a/3213643/4954530
https://stackoverflow.com/a/6168370
Good luck out there!
There are cases where the domain you're looking for could actually be found in the query section but not in the domain section: https://www.google.com/q=www.example.com
This answer would treat that case better.
See this example on regex101.
As you you pointed you only need example.com (write domain then escaped period then com), so use it in regex.
Example
UPDATED
See the answer below
I am trying to test the pathname of the url, checking if pathname starts with privmsg as well as contains one of the words in the selection. And my quantifier is selecting that at least one word must be found.
New RegExp thanks to one of the answers and I extended it more.
var post = /(^\/privmsg\?).+(post|reply){1}(.*)?/;
My urls will look like
/privmsg?mode=post
/privmsg?mode=reply
/privmsg?mode=reply&p=2 //another way
Though we have other modes that I do not want. I need to just get the constant url beginning with privmsg and having at least post or reply in it. Can someone explain what is wrong with my regex string and if I used the quantifier incorrectly.
Problem now is that it is still coming out false...
You need to allow for arbitrary characters between ? and (post|reply) (i.e. mode=). E.g.:
var post = /^\/privmsg\?.+(post|reply){1}/g;
\/
|match any sequence of|
|1 or more characters |
You miss to include something for mode=.
With your regex you will match strings like /privmsg?post.
So alter your regex to include mode=:
^\/privmsg\?.*(post|reply)$
I use this js code to match a hostname from a string:
url.match(/:\/\/(www\.)?(.[^/:]+)/);
This works when the url has protocol:// at the beginning. For example:
This works fine:
var url = "http://domain.com/page";
url.match(/:\/\/(www\.)?(.[^/:]+)/);
But this doesn't:
var url = "domain.com/page";
url.match(/:\/\/(www\.)?(.[^/:]+)/);
I have tried:
url.match(/(:\/\/)?(www\.)?(.[^/:]+)/);
And that matches fine the hostname when it doesn't contain protocol://, but when it does contains it it only returns the protocol and not the hostname.
How could I match the domain when it doesn't contains it?
I used this function from Steven Levithan, it parses urls quite decently.
Here's how you use this function
alert(parseUri("www.domain.com/foo").host)
OK before you have a brain meltdown from #xanatos answer here is a simple regex for basic needs. The other answers are more complete and handle more cases than this regex :
(?:(?:(?:\bhttps?|ftp)://)|^)([-A-Z0-9.]+)/
Group 1 will have your host name. URL parsing is a fragile thing to do with regexes. You were on the right track. You had two regexes that worked partially. I simply combined them.
Edit : I was tired yesterday night. Here is the regex for jscript
if (subject.match(/(?:(?:(?:\bhttps?|ftp):\/\/)|^)([\-a-z0-9.]+)\//i)) {
// Successful match
} else {
// Match attempt failed
}
This
var rx = /^(?:(?:ht|f)tp(?:s?)\:\/\/|~\/|\/)?(?:\w+:\w+#)?(?:(?:[-\w]+\.)+(?:com|org|net|gov|mil|biz|info|mobi|name|aero|jobs|museum|travel|[a-z]{2}))(?::[\d]{1,5})?(?:(?:(?:\/(?:[-\w~!$+|.,=]|%[a-f\d]{2})+)+|\/)+|\?|#)?(?:(?:\?(?:[-\w~!$+|.,*:]|%[a-f\d{2}])+=?(?:[-\w~!$+|.,*:=]|%[a-f\d]{2})*)(?:&(?:[-\w~!$+|.,*:]|%[a-f\d{2}])+=?(?:[-\w~!$+|.,*:=]|%[a-f\d]{2})*)*)*(?:#(?:[-\w~!$+|.,*:=]|%[a-f\d]{2})*)?$/;
should be the uber-url parsing regex :-) Taken from here http://flanders.co.nz/2009/11/08/a-good-url-regular-expression-repost/
Test here: http://jsfiddle.net/Qznzx/1/
It shows the uselessness of regexes.
This might be a bit more complex than necessary but it seems to work:
^((?:.+?:\/\/)?(?:.[^/:]+)+)$
A non-capturing group for the protocol. From the start of the string
match any number of characters until a :. There may be zero or one
protocol.
A non-capturing group for the rest of the url. This part must exist.
Group it all up in single group.