[Music] in order to subscribe to my channel please click here or click here please share comment and like my videos and channel hey guys welcome to SAS word service now this is service now system administrator training and this is part 10 this training has been recorded in Orlando version of ServiceNow before we start the training let me show you the topics of this complete training you in this tenth part of the training we will talk about data import in ServiceNow data import ServiceNow administrators and users can import data into different tables of ServiceNow in this section we will learn about how data can be imported into ServiceNow we will learn about import sets and transform Maps we will learn about colace fields and we will also learn about data policies to import data into ServiceNow need of data import before we learn about importing data into ServiceNow it's important to know why we need to import data into ServiceNow system you might get the requirement from different users to bulk upload the data into ServiceNow in different tables so that you don't need to do manual creation of those rackets as an example you can import the data from Active Directory users and groups data can be imported from Active Directory which get inserted or updated into user or group table respectively you can also import data from HR system some organizations they also import employee's data directly from HR system into ServiceNow you can import knowledge articles so you might need to import knowledge articles from external system or Word documents or any other format you would also get the requirement to import import assets you might need to pull assets information for another system or applications and put them into asset management table or CMDB tables we will learn about CMDB in our next section ways of data import there are different ways you can import data into ServiceNow it is important to know when to use these ways to import data the first way is import XML which only imports xml file that is also for the table you have in ServiceNow so this is the mostly used by developers or administrators to move data from different ServiceNow instances now you will find this feature when you open the list context menu so if you are an administrator and you will you will click on the list context menu you will see this import XML option when you will click on that import XML option it will show new screen to you where you can upload the XML file from your local system now another way is to import data is import which you get on the same list context menu that means import uses the functionality of import sets and transform map which is the core feature of import data into service now when you click on import you will see new screen to upload the file which you have in your system you can only import excel file with this option you will also find an option to download the template provide the data mention the data in that excel sheet and that upload that data back into ServiceNow which will update the tables in ServiceNow and last we have import sets which is the core functionality of ServiceNow for importing data import set is used to is used data from various sources so basically you can import data from various sources and map that data into different ServiceNow tables user with admin or import underscore admin role can import data into ServiceNow ServiceNow has a separate application called system import sets which is used to manage imports in ServiceNow import sets components let's talk about more about import sets and its components so there are various components and inboard sets which process the import of data into ServiceNow first is data source a source of rackets from their data needed need need to be imported it can be a file on a network it can be an attachment in ServiceNow it can be LDAP or it can be any kind of JDBC connection with ServiceNow so data source is something from where exactly you want to import that data it will be a file and in any any any system you have all the possibilities I mentioned then you have import set table import set table is a staging area for rackets to be imported from source before data is inserted or updated into the actual target table it stays in staging table which has fields related to the source data and the name of this import set table is based on the name of the file you try to import so it's kind of a staging table so before putting anything into the real table the actual table you you you put that source data first into this table then that data is being transformed then you have transform map transform map is used to create the relationship between fields of source data and target table as an example I can give you a quick example like you have incident table and you might have some data maybe outside maybe with access or maybe some other data or maybe some other source so it is not mandatory that other other source also has same kind of fields maybe you have ax we have an excel sheet which has just two columns maybe in in place of number it says something else or maybe numbering and the second column you have maybe assigned an assigned to person but here as a external source excel sheet you have maybe assignment now how would you make sure that these two columns should be should should go into the incident table in ServiceNow and should only update number and assigned to now here a peek it comes the transform map where you map the fields and that's how it comes mapping assists so mapping assists provides a visual way to map fields of source data and target data you map them that which field should be mapped to what field of your ServiceNow system mapping can also be done automatically so in transform map when you will create the transform map ServiceNow will give you an option to create these mappings automatically then we have coalesce coalesce is used to trigger a check before importing of data if it is enabled on a field then system will check for existing racket if match found the same racket is updated else new record is inserted coalesce is I would say very important functionality of importing data into ServiceNow and then we have target table that's a main table where you have to import the data so after all these processing the data will be imported in this actual table and that is the target table this is my personal developer instance I will start with showing the example of import XML that is one of the way to import data into ServiceNow so let's say if I go two problem or maybe incident I will go to incident I will come here I have these records maybe I will just here and I have this short description so I will open this racket here a short description is empty what I will do I will export this data and format I will take like this XML this racket so I will click on XML this racket so I got the XML rock racket what I will do I will try to edit that racket now the XML file is opened here in edit mode now you can see whatever fields you have on incident table when I exported that racket into XML it pulled out all all the fields I have you can see all the fields in XML format so in that case I would look for short description and I will try to update it so let's see where we have short description so if I search for short description so here I have short description now what I can do I can remove this and I can copy this I will I will provide a value here and then I can just slash and then I will do this is a demo for importing importing data so I have I'm done with this I will close this what I will do I will save this XML so I should have say what I will do I will just click on save so this XML is saved I will close this I will go back to my instance I'm here now if you remember to import any data via this import XML you have to go to the list view and one thing I want to just make a note here that you can open any list you want it doesn't matter ServiceNow will automatically recognize that table and will insert or update that table it will not import that data into somewhere else so maybe let me show you that as well so let me open problem let's say so I am in problem I'm just doing import XML so here is the feature I will click on choose file I will go to the Downloads mirror my file was downloaded I will select the same file which I have just edited I will click on upload it's done you don't get exactly the message it's done but but it's done and how will you test it I I did it from problem a problem list but I will go to incident and you will see the magic you will see the magic and if I sort this out there is my incident let me do it all maybe let me search with import yeah see this is the same incident which we exported and you can see the data here now this is a demo for importing data now I'm not saying that now why exactly we use this sometime because you use it because sometime you make those changes for the same table you make those changes in different lower instances and then you can import that data into your different environments like maybe production your UAT or different environments their instance or sandbox instance other instances you have that's what you can import data into different instances so if you have done if you have made any changes in one of the load environment and you want to replicate those changes into other instances as well this is how you can do that but as I mentioned it should be a developer or administrator overall you should be an admin then you can do this feature let me show you this import set application which you were talking about so if I go here into application navigator and here I type system import so you just type I am you will get this application called system import sets here you can see we have this like application and you have lot of modules here so starting with load data you have create transform app you can create transform app manually so what I will do I will show you by importing some data but before that I will I will just try to show you some modules that what exactly they are so starting with like load data so if I click here you will see I can load that data into ServiceNow now here is the one which basically creates that staging table and load that data you can you can do from here then we have create transform map so as I was talking about you have to match the fields in that case you can create that transform app from here so you can create new or while you will work on the load data if you will continue to do that you can also do it automatically that is something and do it will automatically follow the same process then you have run transform now in order to load the data you have to import the data and then push that data into this table then you have to click on this run transform this will run the transform but as of now it is asking that create load and imports at first which we have not created yet then you have some like one administration section here you click on data sources which I was talking about that is one of the component here you can define different data sources so for example I can I can show you let's say we have this cs3 may be LDAP users so here you have example LDAP users now this will be directly connected to the LDAP that's one of the sources you have so I can I can just go back so these other examples you can also have some attachments as well if I see you have any CSV may be or may be any other I think I don't I don't think we have any other example where I can show you the attachment but that's okay let's let me show open this one I think this is something ready to the FTP server so you you company must have SFTP server and you can do that this will be done with the help of mid server but that is something you can do as well so then this is about data sources so you can have multiple data sources then you have robust import trans transformers that means you can you can mention that I think this is the something very which I think came later this was not before in the past you can have that robust import set transformers that is something you can create here then we have ETL definitions I think here you can define some definitions then we have transform maps now whatever transform maps you will be creating overall you create data source you create imports at table you also create transform map for each transformation you do you always create a new transform a bird but it is just one time for a particular activity that is something you can do then we have scheduled import so you can that's a good thing so if you have any kind of data source where data is always available maybe every day or or every every six hours or so in that case you can schedule the import that means a system will automatically run the job it will try to import the data which that source has and it's put it into the system that is how that scheduling works so overall you can import the data by its schedules as well frequently as well then you have advanced option here Advanced section where you can see all the import sets now every every data being imported into ServiceNow via this import sets functionality it it is always being trapped but when I say dragged it creates a number it creates a racket and shows you all the details about that particular import whether it was success it was fails it got some errors that is something you can see in import set so whenever if you are getting any functionality if you're getting any data into ServiceNow into into into vayam ports in that case you can always go here and check if anything is failing or everything is passing that is something you can see here and you can try to troubleshoot for any data issues you will get then we have concurrent that means it can be like concurrent import sets you have concurrent imports it's job as well these are some functionalities which ServiceNow came up not not I would say recently but yeah they came up later which was not I think in the beginning when they started with this application then we have progress here you can see the progress of your import that is something you can see you can see transform history that if while transforming the data did you get any issues how many data got inserted updates ignored skipped that's how you can see here runtime completed state when exactly it got started that is something you can see here then we have transform errors if you will get any errors in during your import you can see here which what target record what was the issue that is now you can see so that you can troubleshoot and you will I'm sure in your organization if you will be importing the data I am 100% sure that you might get lot of issues while importing the data I'm Dylan unless you are maybe playing safe from the beginning but sometimes I think it happens because you have a lot of data so sometimes you forget some some kind of prerequisite or proof checks which you have to do as a developer or administrator but that's still okay it's not something end of the world you can you can just try to troubleshoot those issues and just fix them and that's it then you have import log here you will find the import log import locks basically we can delete them so you can see we can cleaning import sets here it kinds of run a job and it clears the old import because we don't need these data it is it does not need for us like we don't need this kind of data to keep the import logs maybe max 7 days or or maybe 10 days or so so you can change this date days but as of now it's just 7 days then we have if I go a little bottom here we have another section import set stable so whenever you will create a new import set table it will automatically be added and shown here in this particular module so here you have import set stable so whatever imports at stable you have in your system you can see here imported stable I have a cleanup as I mentioned you will you will get a cleanup then we have scheduled cleanup as well you can you can schedule those cleanups that every maybe seven all the data before seven days should be deleted that is something you can schedule here so let me let me do one thing let me show you an example by importing some data into ServiceNow so that you will understand the whole process practically but that is important I can show you the PPT but it's it's better to show you practically in the system so you so that you will have right understanding so what I will do I will start with creating an excel sheet first let's example because that said that's the best example I can show you because I don't have any kind of FTP source I don't have any kind of maybe JDBC connection right now or any other network file server I can just show you with the help of excel format so let me create some data and then we will see how exactly you will be importing that data into service now here is my excel sheet now before putting that data before creating that data what I will do we will first select the table in which table we will import this data so I don't want to import that data into ServiceNow incident or change or problem table we will we will try to import this data into into custom table we created so let's let's go here and let's check we had this table if you remember I think we we create this table demo ACL second this one so we will try to import that data into this table that is something we can do so if I go to table and if I say I had I think these two fields right what I will do I will also create a number field because I want to show you the coleus so if I do Auto number I have this ACL table so I think yeah that that's it for now I will save this when I was once I will save this I think I will have that number column as well so if I go here you can see that number column got added automatically now we will have this test one so I will also see these two maybe column name column name yeah we will we will import we will just select this field to be available on this view so we have these three fields what I will do I will just go here and I will try to try to put some columns and data so maybe here I will I will mention string and that's that's see I'm just putting any column name maybe string one and here I am putting a string two that is what I am putting right here so maybe you're not able to see so let me just increase the zoom it so that you can see that so in that case I have a string one string - as of now what I will do I will just put some data so maybe this is maybe just like this I will I will do import one now this is the data I am I'm putting here so I will do import one data one and here input two so that we can recognize whether this data what it wasn't inserted or updated into the same fields so we have this import - now because as us not that table I think just has one or two records so in that case what I will do I will save this I will give it a random name so I will give it as maybe demo demo for import so when didyou demo for import format is Excel we will make it on desktop that is okay and I will click on save now so my this excel sheet is saved now what I will do I will go to my instance here is my instance now in order to import the data what I will do I will go to system import sets so here I have load data so first I will click on load data because I have to first import that data load it into service now staging table first so here I will select this create table you can select the name as well here so maybe I will do demo demo for import that's a table name we are selecting here the file because I don't have any other data source we I will select the file I will select the file directly from my local system so I will go to desktop and here I have that file so I have this demo for import and then I will just click on submit so it says sheet number one header row one so in that case that means header row that's a one row you can define where exactly which one is your header row that is something you can define here and just click on submit once you would click on submit now the import process has started so what it will do it will first process it and create that table for you so you can see it says complete we have been able to import that data into staging table successfully so that says complete success and here I have process - it says inserts to the we have two rackets overall and that's true we had two rackets now the next thing you have to do is create transform map that's the next step so you can see here it says next steps so what I will do because as of now this data is not aware and which is the target table we don't know as of now system does not know about like as a user as an administrator you know about it but system does not know so in that case I will click on create transform map now here you are going to map those fields you are going to make those relationships in that case what I will do here I will just do demo I think we will we can give the name as demo for input that's it that's our transform map name because that's also kind of another record here are some details like active you can just run business rules when importing the data do you want to run the business rule that is something and do and force mandatory fields if you don't do that it can do here copy empty if there is a field empty you still want to copy you can check this box create new record on empty coalesce feels means if coal is filled but as of now we have not selected the coalesce fields and I will show you what is the utilization of colace in that case what I will do I will just go to target table here I will select the target table we are talking about in that case that table is a CL a saying we had a CL demo second so I will select that one and I will click on before clicking on submit here is the deal you have to do auto map matchings I think auto map matchings will not work here we have to go to mapping assist so in that case I will click on mapping assist so before creating that transform map I'm going to mapping assist novi these are the fields for target table we have where we can put the data now this is something we got from our staging table and the fields which we got from our excel sheet those are you will see here I'm sure here we have string one we have a string two you can see we have string two and that's it we had two columns in that extra sheet so in that case I will do one thing I will map this to test one and then this to test you that's it we are not going to put number numbers should come automatically because that's a numbering it should come automatically so what I will do I will click on save we are done with mapping right now relationship is done transform map is also done so you can see I have mapped those fields if you show you as of now nothing is colace even even the number is not coalesce yet you can do that mapping if you want but you should have that data into your excel sheet so as of now I'm not going to do that what I will do I will just check so I will I will just click on transform now because I'm done with and done with table transform map creation mapping assist now I have to transform this data into the actual table so in that case what I will do I will click on transform it will show me the processing now here it will ask me ask me which transform map you want to select in that case it will auto select the transform average we have to run so I will just click on transform map and here is the I set basically import set so I will click on transform and let's it's its success complete and if I go here here's the import set log you can see it says to inserts so two records were inserted no updates nothing in ignored nothing escaped no errors so that means two records were inserted how exactly I can cross-check it I can just go to the same table if I go here you can see we got import one we got import two we got data one we got data - these are the two records which you had in that excel sheet now let's let's let's understand about coal is that how coal is would work here so in that case what I will do I will first add the number number field here number column so you can see the first record because it was created long back that's reason we didn't have numbering at that point of time because we enabled numbering today and now we have those numberings automatically now what happens if I put those numbers in indices in the axial file which I am importing what would happen so here you will learn about the Coley's fields so what I will do I will just put these racquets maybe so maybe a CL this I will just copy this number and I will go to my excel sheet here I have I will just type numbering not number I will type numbering and input two I will just mention that particular so I will copy that so I have this number field here and then I have this one match destination formatting okay it's done so our excel sheet is ready I will save this what do you think if I will import this data when it create new brackets or will it update those rackets so let's take a look so I go here I go to system import sets here I have load data do I need to create a new table so the answer is no because we have already created that table so I will just go to existing and I will select that table and that table will create demo for import I will keep selecting the file I will select my file here and submit it I will click on submit you can see it says to inserts that's okay yet that's still staging table so it has to imp I think insert that data first then I will run the transform now you need to create transform map answer is no the reason behind it because as of now we don't need to map anything basically your transform map was already created but there is one one tweet here because as of now that transform map doesn't have that number field mapping that that that you don't have as of now so let's say I will just click on run if you will click on run it will insert that data so I will come here and I will click on transform that is done and if I come here it inserted two racquets and if I go to cell service we will open it a new tab you can see two more rackets were inserted but this time you can see numbering was done automatically but you mentioned those two numbering in the same in the same rackets the two rules you have so ideally you want them to update not insert so in that case what you need to do you have to put the colace in your transform map how can you do that because you have already created your transform map in that case just go to your application again trigger is system input here you have transform maps the one you just created let's see that here is the one open this go to mapping assist you have to match that number as well and this time you should have that numbering we have this numbering so do you want to match it so I will say yes I will say here basically no because we don't want to match that numbering or or or we want it so I think we should map it so I will just match it and I will save this in that case even if you have masked it and if I import the data it will still insert the rackets do you want to see it let's do that again because that's I think that's how it will give you better understanding maybe let's say I'm changing this to data three let's say I'm I'm just doing it as data three and here as well I'm changing it as data import three let's just to show you so let's see if it imports the data updates the data or insert the data so in that case I will go again to load data here I will select existing table demo for a second that's already selected I will go to demo for import submit go to run transform you can see two were in sir so it captured those two records now it is already selected I will click on transform it will transform the data and if I come here it has inserted again and if I come here and refresh this let's see yep you got six records now so it is still not updating it how it will update it let me show you so in this case I will come here and I will go to my transform Maps open open my transform map which I just created and I will go here now here you have this field mapping and you can you can define the Coley's field now this is this is a really important selection as a developer or administrator because you have to define koulos field which is really you want to make sure that that data you want to update always so in that case numbering is the one which you definitely don't want to insert new right because if it's the same numbering is the same in that case it should not insert new racket it should update the same racket so in that case what I will do I will just double click on it and I will make it colace that's it you're done it is done so you will see the difference of the behavior now so if I go to my excel sheet here so you have these two rackets so what I will do I will again save it I think we didn't change it so let it be like this I think we can just click on save if nothing got changed I will go back to my instance here and this time I will click on load data you will see the difference and but before that I think let's let's change the data let's make it import for so that we will see that only these two rackets will be changed in your instance so we have made it import for data for I have saved it I am going back to my instance let's keep selecting it click on choose file I will select my file and I will just okay I already selected ok no file chosen I will select my file selected I will click on submit now you can see because in a staging table it will always say insert because I don't think it would upgrade it until until that that that particular bracket is there so but that's ok you can click on run transform now if I run transform now you will see that difference and I click on transform it is success and if I go to my import sets here is the difference you can see total was 2 that means it did some two activity two rackets but it was not insert it was update so if I go to this list and if i refresh this you will see that i refresh this and you will see the data changed here import for data for for these two records which were in their excel sheet and they were also mentioned as colace wheels in this system so that's how that's how you can you can define coalesce fields and make sure that you are updating the records it's totally depend on your business case your use case your requirement from your customer but that is how you can define and that's what today's fields is a very I would say very critical functionality of importing data into ServiceNow system let's talk about another functionality that is data policy as of now we have not talked about UI policy because data policy is somehow related to UI policy as well UI policy is something you can put something on the UI and data policy is something that same thing will can be applied into into data when data is being basically imported now it might happen that you might want to put some kind of mandatory checks that this particular field is mandatory in that case we do not want that data to be inserted or updated that means that field should not be blank so in that case what I will do we have this particular table so what I will do I will create a data policy for for this particular table first so maybe I will go to data policy as of now I don't have any data policy for this this table so I will click on new so here is the data policy form and I will just do make test one mandatory so test one field will be mandatory now it will be at data level that means nobody can insert the data even during import so in that case what I will do I will mention I think I don't have to put any kind of condition I can just save this and you can see we have these options here as well apply to import sets apply to soap apply to UI policy as well that's how these data policy works so yes we need to check this box so I think that's the important point in order to apply these data policies you have to keep this checkbox checked that's very important so in that case now I'm going to create that data policy the action what exactly the rule I have I need to apply so I will apply it on one field that is the test one and I will make it mandatory so it is mandatory true and I will click on submit so in that case if that data will not be there in that field our data will not be imported so what I will do now I will go to my excel sheet and I will just do like blank this data so maybe I will I will keep one data and one not so string one is the test one so maybe I will just cancel this one and here maybe I will just do import import data policy and here and here I will not put anything and maybe I can just put here import five here I can put import six maybe but you can see I'm keeping this particular feed blank so let's see what happens so I'm saving this I'm going back to my instance I will go to load data now click on I think it's already selected that's a good thing like it already I think keeps your last selection so I have this one now I will click on choose file I will click on demo for import so the file which we have our click on submit it says to inserts that means I have two data that's ok I will click on run transform let's see then I will click on transform it's done it's succeeded but let's see if I click on transform here import sets it's it's total and it says zero and it is skipped so here you got to is skipped if you click here it says unable to resolve target Coley's values not present you numbering so if I see my data this is not available that means it is very important here so I think we forgot that I think it's it is saying you have to have those Coley's numbering so in that case what I will do I will remove this I want I won't put that number because if you are putting that data then it will ask for that so what I'm doing I'm just closing this I am going to I think this is really good if even if you're getting errors you can just learn new things why you are getting that error maybe I will just do it again I will go for it and I will click on submit run transform let's see what happens in this case and our click on transform success I will go here and it says again colas feels ok it it it cool its values not present that means it is asking for what ideally it should create those rackets it should not if colas fields are not there it should not let me let me check the transform map but why it is why it is ignoring it why it is keep on asking Collins fields it says create new record on Collins I think that's selection we forgot so I think you learned a new selection so college fields non-indexed that's ok cancel for now and here what I will do that that's a very important thing so colas feel that means yes if colas field is not there you want to insert the record you can do that that's kind of mandatory thing you can also apply and so I can just save this so once it is saved what I will do I will load it again I will go here we are not done the indexing but it's it's it's really best practice I would say you should index as well I will select the file again and I will click on submit once I will click on submit let's see it says to inserts I will click on run transform and I will run the transform again success and if I go here let's see what happens import sets awesome so you can see it inserted one racket but not the another one because it says data policy exception where the field is mandatory now this is how you can utilize data policy as well for importing the data so if you want users not to not to import some maybe some junk data where you you have to you have to get mandatory fields in the system you can write data policies into that table under those fields so that you you are getting data into your actual ServiceNow tables so this was the whole whole session about system import sets that how you can import data into ServiceNow so I hope this is really helpful for your organization for you to import data into ServiceNow with different but different scenarios with different functionalities we have in ServiceNow